{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 第7章 网络优化与正则化\n",
    "\n",
    "神经网络具有非常强的表达能力，但当应用神经网络模型到机器学习时依然存在一些难点问题。首先，神经网络的损失函数是一个非凸函数，找到全局最优解通常比较困难。其次，深度神经网络的参数非常多，训练数据也比较大，因此也无法使用计算代价很高的二阶优化方法，而一阶优化方法的训练效率通常比较低。此外，深度神经网络存在梯度消失或爆炸问题，导致基于梯度的优化方法经常失效。\n",
    "\n",
    "目前，神经网络变得流行除了本身模型能力强之外，还有一个重要的原因是研究者从大量的实践中总结了一些经验方法，在神经网络的表示能力、复杂度、学习效率和泛化能力之间找到了比较好的平衡。本章主要介绍神经网络的参数学习中常用的优化和正则化方法。\n",
    "\n",
    "本章内容基于神经网络与深度学习》第7章：网络优化与正则化相关内容进行设计。在阅读本章之前，建议先了解如图7.1所示的关键知识点，以便更好地理解和掌握相应的理论和实践知识。\n",
    "\n",
    "<center><image src=\"https://ai-studio-static-online.cdn.bcebos.com/95762b1e60984b56ac0d7ca32c1bff40ca0edf2de62a4cb598751257fe89e72f\" width=500/></center><br></br>\n",
    "<center>图7.1 网络优化和正则化关键知识点回顾</center><br></br>\n",
    "\n",
    "本章内容主要包含两部分：\n",
    "* 网络优化：通过案例和可视化对优化算法、参数初始化、逐层规范化等网络优化算法进行分析和对比，展示它们的效果，通过代码详细展示这些算法的实现过程。\n",
    "* 网络正则化：通过案例和可视化对$\\ell_{1}$和$\\ell_{2}$正则化、权重衰减、暂退法等网络正则化方法进行分析和对比，展示它们的效果。\n",
    "\n",
    "**提醒**\n",
    "\n",
    "在本书中，对《神经网络与深度学习》中一些术语的翻译进行修正。Normalization翻译为规范化、Dropout翻译为暂退法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 7.1 小批量梯度下降法\n",
    "\n",
    "目前，深度神经网络的优化方法主要是通过梯度下降法来寻找一组可以最小化结构风险的参数。在具体实现中，梯度下降法可以分为批量梯度下降、随机梯度下降和小批量梯度下降(Mini-Batch Gradient Descent)三种方式。它们的区别在于批大小（Batch Size）不同，这三种梯度下降法分别针对全部样本、单个随机样本和小批量随机样本进行梯度计算。根据不同的数据量和参数量，可以选择不同的实现形式。\n",
    "下面我们以小批量梯度下降法为主进行介绍。\n",
    "\n",
    "\n",
    "令$f(\\bm x; \\theta)$表示一个神经网络模型，$\\theta$为模型参数，$\\mathcal{L}(\\cdot)$为可微分的损失函数，$\\nabla_\\theta \\mathcal{L}(\\bm y, f(\\bm x; \\theta))=\\frac{\\partial \\mathcal{L}(\\bm y, f(\\bm x; \\theta))}{\\partial \\theta}$为损失函数关于参数$\\theta$的偏导数。在使用小批量梯度下降法进行优化时，每次选取$K$个训练样本$\\mathcal{S}_t = {(\\bm x^{(k)}, \\bm y^{(k)})}^K_{k=1}$。第$t$次迭代时参数$\\theta$的梯度为\n",
    "\n",
    "$$\n",
    "\\mathbf g_t = \\frac{1}{K}\\sum_{(\\bm x, \\bm y) \\in \\mathcal{S}_t} \\nabla_{\\theta} \\mathcal{L}(\\bm y, f(\\bm x; \\theta_{t-1})),\n",
    "$$\n",
    "\n",
    "其中$\\mathcal{L}(\\cdot)$为可微分的损失函数，$K$为批大小。\n",
    "\n",
    "使用梯度下降来更新参数，\n",
    "$$\n",
    "\\theta_t \\leftarrow \\theta_{t-1} - \\alpha \\mathbf g_t, \n",
    "$$\n",
    "\n",
    "其中$\\alpha > 0$为学习率。\n",
    "\n",
    "从上面公式可以看出，影响神经网络优化的主要超参有三个：\n",
    "\n",
    "1. 批大小$K$\n",
    "2. 学习率$\\alpha$\n",
    "3. 梯度计算$\\mathbf g_t$\n",
    "\n",
    "不同优化算法主要从这三个方面进行改进。下面我们通过动手实践来更好地理解不同的网络优化方法。\n",
    "\n",
    "## 7.2 批大小的调整实验\n",
    "\n",
    "在训练深度神经网络时，训练数据的规模通常都比较大。如果在梯度下降时每次迭代都要计算整个训练数据上的梯度，这就需要比较多的计算资源。另外，大规模训练集中的数据通常会非常冗余，也没有必要在整个训练集上计算梯度。因此，在训练深度神经网络时，经常使用小批量梯度下降法。\n",
    "\n",
    "为了观察不同批大小对模型收敛速度的影响，我们使用经典的LeNet网络进行图像分类，调用paddle.vision.datasets.MNIST函数读取MNIST数据集，并将数据进行规范化预处理。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import paddle\n",
    "\n",
    "# 将图像值规范化到0~1之间\n",
    "def transform(image):\n",
    "    image = paddle.to_tensor(image / 255, dtype='float32')\n",
    "    image = paddle.unsqueeze(image, axis=0)\n",
    "    return image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "方便起见，本节使用第4.5.4节构建的RunnerV3类进行模型训练，并使用paddle.vision.models.LeNet快速构建LeNet网络，使用paddle.io.DataLoader根据批大小对数据进行划分，使用交叉熵损失函数及标准的随机梯度下降优化器paddle.optimizer.SGD。RunnerV3类会保存每轮迭代和每个回合的损失值，可以方便地观察批大小对模型收敛速度的影响。\n",
    "\n",
    "通常情况下，批大小与学习率大小成正比。选择批大小为16、32、64、128、256的情况进行训练。相应地，学习率大小被设置为0.01、0.02、0.04、0.08、0.16。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Cache file /home/aistudio/.cache/paddle/dataset/mnist/train-images-idx3-ubyte.gz not found, downloading https://dataset.bj.bcebos.com/mnist/train-images-idx3-ubyte.gz \n",
      "Begin to download\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "item  384/2421 [===>..........................] - ETA: 17s - 8ms/ite\n",
      "item 2421/2421 [============================>.] - ETA: 0s - 5ms/item"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Download finished\n",
      "Cache file /home/aistudio/.cache/paddle/dataset/mnist/train-labels-idx1-ubyte.gz not found, downloading https://dataset.bj.bcebos.com/mnist/train-labels-idx1-ubyte.gz \n",
      "Begin to download\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "item 8/8 [============================>.] - ETA: 0s - 12ms/item"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Download finished\n",
      "W0716 15:38:02.097203   150 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 10.1\n",
      "W0716 15:38:02.100914   150 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Train] Training done!\n",
      "[Train] Training done!\n",
      "[Train] Training done!\n",
      "[Train] Training done!\n",
      "[Train] Training done!\n"
     ]
    }
   ],
   "source": [
    "import paddle.io as io\n",
    "import paddle.optimizer as optimizer\n",
    "import paddle.nn.functional as F\n",
    "\n",
    "from nndl import RunnerV3\n",
    "from paddle.vision.models import LeNet\n",
    "from paddle.vision.datasets import MNIST\n",
    "\n",
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "\n",
    "# 准备数据\n",
    "# 确保从paddle.vision.datasets.MNIST中加载的图像数据是np.ndarray类型\n",
    "paddle.vision.image.set_image_backend('cv2')\n",
    "train_dataset = MNIST(mode='train', transform=transform)\n",
    "# 迭代器加载数据集\n",
    "# 为保证每次输出结果相同，没有设置shuffle=True，真实模型训练场景需要开启\n",
    "train_loader1 = io.DataLoader(train_dataset, batch_size=16)\n",
    "\n",
    "# 定义网络\n",
    "model1 = LeNet()\n",
    "# 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "opt1 = optimizer.SGD(learning_rate=0.01, parameters=model1.parameters())\n",
    "# 定义损失函数\n",
    "loss_fn = F.cross_entropy\n",
    "# 定义runner类\n",
    "runner1 = RunnerV3(model1, opt1, loss_fn, None)\n",
    "runner1.train(train_loader1, num_epochs=30, log_steps=0)\n",
    "\n",
    "model2 = LeNet()\n",
    "train_loader2 = io.DataLoader(train_dataset, batch_size=32)\n",
    "opt2 = optimizer.SGD(learning_rate=0.02, parameters=model2.parameters())\n",
    "runner2 = RunnerV3(model2, opt2, loss_fn, None)\n",
    "runner2.train(train_loader2, num_epochs=30, log_steps=0)\n",
    "\n",
    "model3 = LeNet()\n",
    "train_loader3 = io.DataLoader(train_dataset, batch_size=64)\n",
    "opt3 = optimizer.SGD(learning_rate=0.04, parameters=model3.parameters())\n",
    "runner3 = RunnerV3(model3, opt3, loss_fn, None)\n",
    "runner3.train(train_loader3, num_epochs=30, log_steps=0)\n",
    "\n",
    "model4 = LeNet()\n",
    "train_loader4 = io.DataLoader(train_dataset, batch_size=128)\n",
    "opt4 = optimizer.SGD(learning_rate=0.08, parameters=model4.parameters())\n",
    "runner4 = RunnerV3(model4, opt4, loss_fn, None)\n",
    "runner4.train(train_loader4, num_epochs=30, log_steps=0)\n",
    "\n",
    "model5 = LeNet()\n",
    "train_loader5 = io.DataLoader(train_dataset, batch_size=256)\n",
    "opt5 = optimizer.SGD(learning_rate=0.16, parameters=model5.parameters())\n",
    "runner5 = RunnerV3(model5, opt5, loss_fn, None)\n",
    "runner5.train(train_loader5, num_epochs=30, log_steps=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可视化损失函数的变化趋势。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnXd4VVX2v9+VXgkJoZfQ+yAKwgBDUUSCIgjypapUEQsq/FBgLDCOzth1lCLYsDCDioqoCCIQQXrAgIAECAgEQgk9BELK+v1xC/eGlJt6w81+n+c8uWfXdc69WWefffZZH1FVDAaDwVA+8HK3AQaDwWAoPYzTNxgMhnKEcfoGg8FQjjBO32AwGMoRxukbDAZDOcI4fYPBYChHGKdvAEBE6oqIioiPi+XnicgLJW1XQRCRv4vI+3nkjxCRX4vQvv2YRaSziMQ75DURkTgRuSAij4lIoIh8JyLnROTLwvZZUhT1XJQE+f0GReRPEbmttO3yNFz6BzcYrgdU9V+2zyJSFzgA+KpqRgn0tQZo4pD0FLBKVVtb+78PqApUKon+80NEFGikqvtKu29D2caM9A2G4iEK2Jltf09hHL6rd1uGq5hz5jrG6ZdRRKSGiHwlIidF5ICIPOaQN11EForI59bphK0icoNDfjMRiRGRsyKyU0T6OOQFisjrInLQOvXwq4gEOnQ9TEQOiUiyiDxdAHsfEJF9InJaRBaLSA1ruojImyJyQkTOi8jvItLSmneHiOyyHsMREZmUS9sHRaSN9fMw6xRAC+v+aBFZ5HBePrNWW239e1ZEUkSkg0N7r4nIGet57ZXHMd1oPbcXRORzIMAhr5uIJFo/rwRuAWZY+/of8BwwyLo/2lpulIj8Ye17mYhEObSnIvKIiOwF9lrTmorIcus5jReRgQ7l54nITBH5wWrfRhFpYM2zHfs2a/+Dcj9EmWH9HewWke4OGSNEZL+17QMiMiyXBtqJyHrrby3J2p5ftuMaJyJ7rWVmiohY87yt30WyiOwH7sztu8ihX9v/wGcich4Y4Wrdco+qmq2MbVguxluwOA4/oD6wH+hpzZ8OpAMDAF9gEtapDOu2D/i7te6twAWgibXuTCAGqAl4Ax0Bf6AuoMB7QCBwA5AGNMvFxnnAC9bPtwLJwE3Wtt4BVlvzelqPpSIgQDOgujUvCehs/RwO3JRLX58A/8/6eS6QADzkkDfB4bx8Zv1sOx4fh3ZGWM/bA9Zjfwg4CkgOffoBB4EJ1nM6wFrXdszdgESH8jHAGId9uy3W/b7W76UZlmnVZ4B1DvkKLAcirOc/GDgMjLSWv9F6jps7nP9TQDtr/nxgQbb2GubxGxsBZDgc3yDgnLX/YOA8V38z1YEWubTTBvir1Ya6wB/AE9ns+N76/dcBTgLR1rxxwG6gtrXfVdm/s2x9/Qnclu1/4G4s/y+B7v6/vV42M9Ivm9wMVFbV51X1iqrux+KMBzuU2aKqC1U1HXgDyyj0r9YtBHjJWnclln+6ISLiBYwCHlfVI6qaqarrVDXNod1/qOolVd0GbMPi/PNjGPChqm61tjUV6CCWefV0IBRoisW5/qGqSdZ66UBzEamgqmdUdWsu7f8CdLV+7gz822G/qzXfVQ6q6nuqmgl8jMWhVc2h3F+xOMO3VDVdVRcCmwvQT3bGAf+2Hn8G8C+gteNo35p/WlUvAb2BP1X1I1XNUNXfgK+A/3Mo/42qbrK2Nx9oXUCbTnD1+D4H4rk62s4CWopIoKomqerOnBpQ1S2qusFq45/AHK5+NzZeUtWzqnoIi2O32TnQ2v9hVT2N5XstCOtVdZGqZlnPmcEFjNMvm0QBNay3w2dF5CyWkbujczps+6CqWUAiUMO6Hbam2TiIZWQfieXikJBH38ccPqdiuYDkRw1rHzZ7UrCMQmtaLzozsNxhnBCRuSJSwVr0HuAO4KCI/OI4BZONX4DOIlIdywj9C6CT9aISBsS5YKMN+/Gpaqr1Y07HWAM4oqqOEQkP5lDOVaKA/zh8n6ex3PnUdChzOFv59tl+A8OAag5lCvNdOZLT8dVQ1YtYRv7jgCTrFFLTnBoQkcYi8r2IHLNOs/wLy+/MkdzsrIHzMRf0/B7Ov4ghO8bpl00OAwdUtaLDFqqqdziUqW37YB3B18IyVXEUqG1Ns1EHOIJleuAy0KCY7T2KxUnZ7AkGKln7RFXfVtU2QHOgMfCkNX2zqvYFqgCLsDjza1DLCpRUYDyWaaPzWBzJWODXbBc4e7UiHlMSUNM2/2ylThHaOww8mO07DVTVdQ5lNFv5X7KVD1HVh4pgQ3ZyOr6jAKq6TFV7YLkT2o3lTjMnZlvzG6lqBSyDE8mlbHaScPgdU/Dza0IEFwLj9Msmm4ALIjJZLA9evUWkpYjc7FCmjYj0F8uqhSewzL9vADZicZBPiYiviHQD7sIy35sFfAi8IZYHxd4i0kFE/Ito7/+AkSLS2trWv4CNqvqniNwsIu1FxBe4iOWikyUifmJ5KBtmnaI6j2VKITd+AR7l6lROTLb97Jy0tle/kMe0Hsuc92PW89gfy/x5YXkXmCpXH0CHicj/5VH+e6CxiNxn7d/Xei6budjfcfI/9ipcPb7/w/K8YYmIVBWRvtaLdxqQQu7fTSiW7y7FejdQkIvSF9b+a4lIODClAHUNhcQ4/TKIdb65N5a5zwNYRujvY5nKsPEtllvwM8B9QH/r3OwVLE6+l7XeLOB+Vd1trTcJ+B3L/PRp4GWK+DtQ1Z+BZ7HMOSdhuZOwPX+ogGWUeAbL7fsp4FVr3n3An9ZpgXFYpi9y4xcsDmZ1LvvZbUoFXgTWWqdH/lrAY7oC9MfywPM0lnP9dUHayNbeN1jO9QLr8e7A8h3lVv4CcDuW83gUy53Ny1gelLvCdOBj67EPzKXMRqARlt/Ji8AAVT2F5fcw0drvaSxz9Lk580nAUCyLBd4DPnfRPqzll2F5drSVIpxfg+uI85Se4XpARKZjWZlxr7ttMRgM1xdmpG8wGAzlCOP0DQaDoRxhpncMBoOhHGFG+gaDwVCOKHNBiiIjI7Vu3bruNsNgMBiuK7Zs2ZKsqpXzK1fmnH7dunWJjY11txkGg8FwXSEiLr3RbKZ3DAaDoRxhnL7BYDCUI4zTNxgMhnKEcfoGg8FQjjBO32AwGMoRxukbDAZDOaLMLdk0XB9kZWWRnJzM2bNnyczMdLc5BoNH4+3tTcWKFYmMjMTLq2hjdeP0DYUiMTEREaFu3br4+vrirMVhMBiKC1UlPT2d48ePk5iYSJ06RdHy8aDpndTUVG655Rbmz5/vblPKBRcvXqRmzZr4+fkZh28wlCAigp+fHzVr1uTixYtFbs9jnH5AQAC//PIL8fHx7jal3FDU20yDweA6xfX/5jH/tV5eXoSHh3P69Gl3m2IwGAxlFpecvohEi0i8iOwTkVx1LEXkHhFREWnrkDbVWi9eRHoWh9G5ERERYZy+wWAw5EG+Tl9EvIGZWPQ8mwNDRKR5DuVCgcex6G7a0ppj0fhsAUQDs6ztlQjG6RtcoVu3bowZM8YtfU+fPp2GDRsWe7siwmeffVbs7ZYl3Pm9eRKujPTbAftUdb9VLHoB0DeHcv/EItx82SGtL7BAVdNU9QCwz9peidCmTRvq1atXUs0bDE74+Pgwb948d5sBQFJSEgMGDCj1fp944gnat29PUFAQPj65LwZcvXo1t956KyEhIYSEhHDzzTdz4MCBUrT0Wvbs2UPPnj0JCgoiMjKScePGufSgdMmSJbRu3Rp/f3/q1q3LG2+84ZSflJTEsGHDaNGiBT4+Ptx2220ldQiFwhWnXxM47LCfaE2zIyI3AbVV9YeC1i1OZs2axezZs0uqeYOhzFKtWjUCAgJKvd/MzEyGDh3Kww8/nGuZpUuX0qtXL7p168a6deuIi4vjueeeIygoqFhtuXLlistlU1JS6N69Oz4+Pqxbt44vvviCpUuXMnr06DzrxcbG0rdvX3r16kVcXBzTp0/n73//O++++669TFpaGhEREUycOLHMOXzAsgY0rw0YALzvsH8fMMNh3wuIAepa92OAttbPM4B7Hcp+AAzIoY+xQCwQW6dOHTWUfXbt2uVuEwpN165ddeTIkTp58mStVKmShoaG6gMPPKCXLl2yl/npp5+0a9euGh4erhUqVNAuXbroxo0b7flRUVEKOG02YmNjtWfPnhoaGqrBwcF6880364YNG1RVddq0adqgQQNdtGiRNmnSRIOCgrRr1666Z8+ePG1es2aNduzYUUNCQjQkJERbtWqlS5cutecD+umnn9r7yG4boMOHD3c6vo4dO2pAQIDWqFFDR4wYocnJyYU+px999JF6e3tfk56Zman16tXTKVOmFLptG127dtXRo0c77Y8aNUqfeeYZrVatmlatWtXltubMmaMBAQF69uxZe9r333+vgO7fvz/XekOGDNEOHTo4pU2aNEmjoqJyLD98+HDt3r27y3blR17/d0Cs5uPPVdWll7OOALUd9mtZ02yEAi2BGOt67WrAYhHp40Jd24VnLjAXoG3btoUW7Z07dy5vvPEGu3btMssJS5kNGzZyyg3PUypFRPDXv7YvcL2FCxcyaNAg1qxZw759+xg9ejTBwcG8+eabgGUk+PDDD3PDDTeQkZHBm2++SXR0NHv37qVSpUps3ryZ6tWr8/rrrzNo0CB7uzt37qRLly706dOHlStXEhYWRmxsLFlZWfYySUlJzJ49m/nz5+Pj48OoUaMYNWoUa9asydHWjIwM+vTpw4gRI+zTSTt27Mh1pDxp0iTGjRtn39+8eTP9+/fnlltuAWDlypX07duXl19+mXnz5nH27Fmeeuop+vfvT0xMDCJCTEwMt9xyC6tWraJbt24FPr82tm7dyoEDB6hVqxZdunRh9+7d1K9fn8mTJ9OvX79Ct2vjiy++YNiwYaxYscL+Zvj06dP5xz/+YRtQ5sjatWvp0KEDYWFh9rTbb78dLy8v1q5dm+s08dq1a6+5G4iOjua1114jMTGRWrVqFfmYShpXnP5moJGI1MPisAcDQ22ZqnoOiLTti0gMMElVY0XkEvBfEXkDqAE0AjYVn/nOpKamEh8fz7lz5wgPDy+pbgweQEREBO+++y7e3t40a9aMF154gccee4wXXniB4ODgaxzS3Llz+eqrr1i6dCnDhg2jcmWLKl1YWBjVqlWzl3vppZdo2LAh8+fPtw88GjVq5NRWWloan376qb2Np556iiFDhnD58uUcp2guXLjAmTNn6NOnj72t7G06Yps3Bzh8+DBjx45lwoQJDB8+HIDnn3+exx57jPHjx9vrfPzxx0RFRbFt2zZat25NUFAQTZo0KfIUTEJCAgDPPPMML7/8Mu3ateP777/nnnvuYdmyZfTo0aNI7VevXp1Zs2Y5DfIiIyNp0qRJnvWSkpKcvjcAX19fIiIiSEpKKlA9235SUpJnOH1VzRCRR4FlgDfwoaruFJHnsdxOLM6j7k4R+QLYBWQAj6hqiQVqiYiIAOD06dPG6ZcyhRltu5N27drh7X11IVmnTp1IS0sjISGBVq1aceDAAZ577jnWr1/PiRMnyMrKIjU1lYMH81ak27JlC9HR0XneadaoUcPu8G37qsqJEydyfMU+PDycMWPG0LNnT2699Va6du1Kv3798nVsKSkp3HXXXXTo0IGXX37Znr5582Y2bNjAjBkzrqmzd+9eWrduTbt27di9e3ee7buC7Q5nzJgxjB07FoDWrVuzYcMG3nnnnSI7/TZt2lxzrh999FEeffTRIrXrybg0B6KqS1S1sao2UNUXrWnP5eTwVbWbqsY67L9orddEVX8sPtOvxdHpGwxFoXfv3hw6dIiZM2eyYcMG4uLiqFKlSoEeFuaGn5+f074tjIXjFFB23nvvPbZs2UKPHj345ZdfaNmyJXPmzMm1fFZWFkOGDMHX15fPPvvMKVRGVlYWkydPJi4uzmnbu3cvvXr1KuLROVO9enUAWrRo4ZTeokWLfC+grhAcHFxou44dO+aUlp6ezunTp+02u1rv+PHj9rzrAY+a+DZO3+AqmzdvdooOum7dOvz9/WnQoAGnTp1i165dTJkyhZ49e9K8eXMCAgI4ceKEUxt+fn7XRBht06YNK1asyNOBF5aWLVsyceJEfvzxR0aPHs3cuXNzLTtp0iS2bdvGd999d80UTdu2bdm5cycNGza8ZrNNCxUXbdu2JTAw8Jq7hvj4eOrWrVusfRWETp06sX79es6fP29PW758OVlZWXTq1CnPesuWLXNKW7p0KVFRUdfF1A54mNOvUaMG0dHRhIaGutsUQxnn1KlTPPLII/zxxx/88MMPPPvsszz44IMEBwcTHh5O5cqVee+999izZw/r169nyJAhBAYGOrVRr149Vq1axdGjR0lOTgYs8/N79+5l2LBhxMbGkpCQwJdffsn69esLbeu+ffuYPHkyv/76KwcPHmT9+vWsWbOG5s2veUcSgHnz5jFr1izef/99AI4dO8axY8c4d+4cYJnT//bbb5k4cSJxcXEkJCTYlyteunQJgE2bNtG0aVM2bcr7Edy+ffuIi4vj0KFDAPa7hpSUFMDyfGH8+PHMnDmT//73v+zbt4+33nqL7777jscff7zQ5yQvZsyYQdOmTfMsM3ToUCIjIxk6dCjbtm1j1apVPPLIIwwaNMjpIW7Tpk2dpsEmTJjApk2bePrpp9m9ezcff/wx77zzDlOmOAcqsJ2H06dPk5KSYt8vE7iyxKc0tzZt2ri6esngRjxhyeakSZM0IiJCQ0JCdPTo0ZqammovExMTo61atVJ/f39t3LixLly4UBs0aKDTpk2zl/nxxx+1adOm6uvr67Rkc+PGjdq9e3cNCgrSkJAQbd++vX25p23JpiNr1qxRQA8cOJCjvUePHtV+/fppzZo11c/PT6tXr65jxoxxWm6Iw5LN4cOH57tkc/Xq1dq9e3cNCQnRoKAgbdq0qT7++OOanp6uqqqrVq1SQFetWpXvucypL8d6GRkZ+uyzz2rNmjU1KChI27Rpo99++61TO8OHD8912aNjX9mXbDru27AtWc2P3bt3a48ePTQwMFAjIiJ07NixmpKS4lQGcPrOVS1LO1u1aqV+fn5ap04dff31169pO6dz4opN+VEcSzZF81jW5A7atm2rsbGx+Rc0uJU//viDZs2audsMg4fQpUsXmjVrludzCkPe/3ciskVV2+aY6YBHTe+AZd7z73//u7vNMBgMLnLmzBni4+P517/+5W5TygUep5yVkpLCkSPXvP9lMBjKKOHh4fYVMIaSx+NG+ibSpsFgMOSOcfoGg8FQjjBO32AwGMoRHjenf8stt1CzZolFbzYYDIbrGo9z+g899JC7TTAYDIYyi8dN78DVF84MBoPB4IzHOf358+fj5+dnfy3cYDAYDFfxOKcfHBxMRkaGeZhryBUjjH59YoTRiwePc/om0qahtCjvwuiXLl2id+/e1KlTh4CAAKpWrcrdd9/Nrl277GUOHTrEgw8+SKNGjQgMDKRWrVqMHDmyTLxAWVLC6N9++y133HEH1apVIygoiBYtWvCf//ynzEw5e5zTt4mnnDlzxs2WGAylhzuE0UWEHj168MUXXxAfH88PP/xARkYG3bt35/Lly4AlhPLFixd566232LFjBwsWLGDnzp1ER0dfE5a6qJQVYfSYmBg6dOjAN998w44dO3jyySeZOnUqr7zySqGPrVhxJSobEA3EA/uAKTnkjwN+B+KAX4Hm1vS6wCVrehzwbn59FTXKZmJiogI6Z86cIrVjyBtPiLJphNGLTxjdRlxcnAIaFxeXa5ktW7YooNu3by9Q29ejMLqN8ePH60033eSyfblRHFE28x3pi4g3MBPoBTQHhohI9kDe/1XVv6hqa+AVwPF+J0FVW1u3cZQwlSpV4qGHHspXSs5QvKQfPUPa/hOlvqUfLdwd3cKFCzl16hRr1qxh/vz5LFq0iKlTp9rzbcLo69evZ926dTRq1Ijo6GhOnToFWERYvL29eeutt0hKSrLrqtqE0cPDw1m5ciW//fYbEyZMyFUYfd26dVy4cIFRo0blaqtNGL19+/Zs3bqVrVu3Mn369DyF0W02JSUlsXjxYnx8fK4RRh88eDDbt29n0aJF/Pnnn/Tv398+BeEokO4qFy5c4P3336dmzZp5aviePXsWoMj6u2ARRj958iQrVqxg+fLlgOW5iaNSWE7kJ4yeV73o6GintOjoaA4ePEhiYmKu9c6ePVtola/ixpV1+u2Afaq6H0BEFgB9sejeAqCq5x3KB2MZWbiFgIAAZs2a5a7uDdcJRhi9+ITRJ0+ezMyZM7l48SLNmjUjJiYm13opKSlMnDiRe+65hwYNGuTbdn5cD8LoMTEx/O9//+Prr7/O93hKA1ecfk3gsMN+InCNCraIPAJMBPyAWx2y6onIb8B54BlVXZND3bHAWCBHYeiCkpGRQVpaWpm5spYHfGtcX0L0Rhi9+ITRn3zyScaMGcORI0d47bXX6NevH+vWrbtGwe7ixYv06dMHHx8fPvjgA5fazo+yLoy+YcMG7r77bqZPn85dd93lbnOAYnyQq6ozVbUBMBl4xpqcBNRR1RuxXBD+KyIVcqg7V1Xbqmpbx3+GwnLTTTdx3333FbkdQ/nFCKO7TmRkJI0aNaJbt258/fXX/Pnnn8yfP9+pzLlz5+jZsycXL17k559/dppWKQplWRg9JiaGHj168NRTT/H0008Xys6SwBWnfwSo7bBfy5qWGwuAuwFUNU1VT1k/bwESgMaFM9V1wsPDzZJNQ54YYfSSE0ZXVfvqHYDk5GT784Tly5dTsWLFIvdRVEpaGP2HH37gjjvusK/uKUu44vQ3A41EpJ6I+AGDgcWOBUTEcYLxTmCvNb2y9UEwIlIfaATsLw7D88JE2jTkhxFGL7owekxMDLNnz2bbtm0cOnSItWvXMmDAALy8vOjfvz9gmefu0qULIsLHH39Mamqq3Z7iuGvKCXcLo3/55Zf069ePSZMmMWzYMPvxnjx5svgPtjC4ssQHuAPYg2Wk/rQ17Xmgj/Xzf4CdWJZlrgJaWNPvcUjfCtyVX1/FIYw+atQorVmzZpHbMeSOJyzZNMLoRRNG37Bhg3bt2lUjIiLsIuHDhg3TnTt32st89NFHuYqEO7btScLouYnF53d8rmCE0XPhySefZObMmaSmphaTVYbsGGF0Q3FihNFdoziE0T0utDJAz549CQsLIysrK89VFAaDwf3YhNG/+eYbd5tSLvBIp3/bbbdx2223udsMg8HgAkYYvXTxyGFweno6iYmJTisIDAaDweChTv+XX36hdu3aea48MBgMhvKIRzp9E17ZYDAYcsYjnX6lSpUA4/QNBoMhOx7p9M1I32AwGHLGI51+SEgIPj4+xukbDAZDNjxyyaaI8Oabb3LjjTe62xSDwWAoU3ik0wfKTGhVg8FgKEt45PQOWMQjXI0HbihfdOvWjTFjxril7+nTp9OwYcNib1dE+Oyzz4q93bKEO783T8Jjnf7DDz/M0KFD3W2GwYPx8fFh3rx57jYDsESzHDBggFv6XrRoEe3btycwMJCwsDC6dOlCSkpKjmVHjBiBiPDCCy+UspXXsmfPHnr27ElQUBCRkZGMGzeOixcv5ltvyZIltG7dGn9/f+rWrcsbb7yRa9ljx45RrVo1RCRPOcXSxGOdfnh4OGfOFE4/1WC43qhWrVqOUoslzQcffMD999/PsGHD2Lp1K5s2bWL8+PFOqmQ25s2bx++//06NGjVKxJaChGpOSUmhe/fu+Pj4sG7dOr744gt7eOm8iI2NpW/fvvTq1Yu4uDh7vPx33333mrJZWVkMGzaMdu3aFfhYShRXQnGW5lYcoZVVVR9//HGtUKFCsbRluBZPCK08efJkrVSpkoaGhuoDDzygly5dspf56aeftGvXrhoeHq4VKlTQLl262MMjq6pGRUVdEzrXRmxsrPbs2VNDQ0M1ODhYb775Zt2wYYOqXg2tvGjRIm3SpIkGBQVp165ddc+ePXnavGbNGu3YsaOGhIRoSEiItmrVSpcuXWrPxyG0si20cPbNMbTyTz/9pB07dtSAgACtUaOGjhgxQpOTkwt0Hs+dO6ehoaH67rvv5lt2586dWrVqVd2zZ49GRUXpP//5zwL1pZpzaOVRo0bpM888o9WqVdOqVau63NacOXM0ICDAKTz1999/r4Du378/13pDhgzRDh06OKVNmjQpx7DJ06dP1+joaF25cqUCevjwYZfty43iCK3ssQ9yIyIiOH/+POnp6fj6+rrbHI/n8sYEMk/nf2tc3HhHBBPQvuAC2wsXLmTQoEGsWbOGffv2MXr0aIKDg3nzzTcBy0jw4Ycf5oYbbiAjI4M333yT6Oho9u7dS6VKldi8eTPVq1fn9ddfZ9CgQfZ2d+7cSZcuXejTpw8rV64kLCyM2NhYJyWtpKQkZs+ezfz58/Hx8WHUqFGMGjWKNWuukY8GLJrPffr0YcSIEfbppB07duQqPj5p0iTGjRtn39+8eTP9+/e3q1etXLmSvn378vLLLzNv3jzOnj3LU089Rf/+/YmJiUFEiImJ4ZZbbmHVqlV069Ytx35++uknLly4QGBgIG3btuXw4cM0a9aMf/7zn3Tu3NleLjU1lYEDB/Lqq6/mKeheGL744guGDRvGihUr7Cpm06dP5x//+IdNCyRH1q5dS4cOHZxkG2+//Xa8vLxYu3atk5BK9nrZ7waio6N57bXXSExMtKtnrVq1ivfee4+tW7eya9euoh5mseKS0xeRaCxCKd7A+6r6Urb8ccAjQCaQAoxV1V3WvKnAaGveY6rqrDVWQthe0Dp79izFobtr8CwiIiJ499138fb2plmzZrzwwgs89thjvPDCCwQHB9OvXz+n8nPnzuWrr75i6dKlDBs2zP6bCgsLo1q1avZyL730Eg0bNmT+/Pn2sN7ZHV1aWhqffvqpvY2nnnqKIUOGcPny5RynaC5cuMCZM2fo06ePva28nGdISIhd9vDw4cOMHTuWCRMmMHz4cMCinPXYY48xfvx4e52PP/6YqKgotm3bRuvWrQkKCqJJkya5XlgAEhISAPj73//Oa6+9RuPGjfm4HXBXAAAgAElEQVToo4/o3r07cXFxdmWvRx55pMR0q6tXr86sWbOcQqhHRkbmKxqflJTk9L0B+Pr6EhERQVJSUoHq2faTkpKoVasWx48f59577+Xjjz+mSpUq15/Tt8odzgR6AInAZhFZbHPqVv6rqu9ay/cB3gCiRaQ5FnnFFkAN4GcRaayqzsKiJUCPHj347LPP8vzRGoqPwoy23Um7du2c5p07depEWloaCQkJtGrVigMHDvDcc8+xfv16Tpw4QVZWFqmpqRw8eDDPdrds2UJ0dHSeOg41atRwGojUqFEDVeXEiRPUqVPnmvLh4eGMGTOGnj17cuutt9K1a1f69euXr2NLSUnhrrvuokOHDrz88sv29M2bN7NhwwYnGUAbe/fupXXr1rRr1y7f1W+2u5epU6cyePBgAG666SZiYmJ49913efvtt5k/fz5r165l69atebZVWNq0aXPNuX700UfdumR72LBh3H///WU2vLsrD3LbAftUdb+qXsEifN7XsYCqnnfYDcYyh4i13AK1CKQfAPZZ2ytxmjRpwrBhwwgODi6N7gweRu/evTl06BAzZ85kw4YNxMXFUaVKlWLRdfXz83PaFxGAPMXU33vvPbZs2UKPHj345ZdfaNmyZZ4qU1lZWQwZMgRfX18+++wzex+2vMmTJxMXF+e07d27l169erl8HNWrVwegRYsWTunNmze3XxyXL1/Ovn37qFixIj4+Pvj4+HDw4EGmTZtWLA+eC/v/Xb16dY4dO+aUlp6ezunTp+3H5Wo9mxaArd6KFSt49dVX7cfbvXt3AOrWrcuDDz5YKHuLE1emd2oChx32E4H22QuJyCPARMAPuNWh7oZsdWsWytICkpqaSmxsLI0bN77mdsxg2Lx5M5mZmfbR/rp16/D396dBgwacOnWKXbt2sWTJEnr27AlAYmIiJ06ccGrDz8/PPo9so02bNqxYsaJEVNtatmxJy5YtmThxIuPGjWPu3Lm5OpFJkyaxbds2Nm3adM3dbtu2bdm5c2eR3xewzdvv3r3bad4/Pj6erl27AvDiiy8yadIkp3o9e/bknnvucasD7NSpE48//jjnz5+nQoUKgOUClZWVRadOnfKst2zZMp577jl72tKlS4mKirLP5//+++9OdTZv3syoUaNYtmxZmZAYLbZfparOVNUGwGTgmYLUFZGxIhIrIrHFpRh/9OhRunbtyvLly4ulPYNncerUKR555BH++OMPfvjhB5599lkefPBBgoODCQ8Pp3Llyrz33nvs2bOH9evXM2TIEAIDA53aqFevHqtWreLo0aMkJycDlvn5vXv3MmzYMGJjY0lISODLL79k/fr1hbZ13759TJ48mV9//ZWDBw+yfv161qxZY58zz868efOYNWsW77//PmBZK37s2DHOnTsHWOb0v/32WyZOnEhcXBwJCQn25YqXLl0CYNOmTTRt2jRPTYoGDRowcOBA/vGPf/DDDz+wd+9epk6dyu7du3nooYcAqFmzpv1iZdt8fX2pUqXKNXcIxcWMGTNo2rRpnmWGDh1KZGQkQ4cOZdu2baxatYpHHnmEQYMGOT3Ebdq0qdM02IQJE9i0aRNPP/00u3fv5uOPP+add95hypQp9jLZj9fWXpMmTUpsuWpBcMXpHwFqO+zXsqblxgLg7oLUVdW5qtpWVdsW10NXE2nTkBcDBgwgNDSUv/3tbwwePJjevXvz0kuW9QleXl58+eWX9vn9ESNG8MQTT1xz2//666+zZcsW6tata5+j/8tf/kJMTAwnT56ka9eutG7dmtdffz3HdeuuEhwczN69exk8eDCNGzfmnnvuoWPHjjnOyQPExMSQlpZGz549qV69un17/PHHAbjllltYuXIl27dvp3PnzrRq1YoJEyYQGhpqX+mWmppKfHw8qampedr20Ucfcc899zBy5Ehuuukmfv31V1asWJGv083OiBEjqFu3boHq5EZycjLx8fF5lgkJCeHnn3/mypUrdOjQgQEDBnD77bfzwQcfOJWLj4+3X9ABbr75ZhYtWsT333/PDTfcwHPPPceLL77otFqqzJPfmk4sU0D7gXpYpm62AS2ylWnk8PkurOtFsTzA3Qb4W+vvB7zz6q+41ulnZGSoiOhzzz1XLO0ZnLme1+kbyh6dO3fWsWPHutuMMk+prNNX1QwReRRYhmXJ5oequlNEnrd2shh4VERuA9KBM8Bwa92dIvIFsAvIAB7RUli5A+Dt7U3FihXNSN9gKOOcOXOG+Ph4vvnmG3ebUi5waZ2+qi4BlmRLe87h8+N51H0ReLGwBhaFiIgI4/QNhjJOeHi4fQWMoeTx2DdyAebMmUNkZKS7zTAYDIYyg0c7fdv6WIPBYDBY8Ngom2BZL/v999+72wyDwWAoM3i0058zZ4493ojBYDAYPNzpR0REcObMmTxfbzcYDIbyhMc7fVW1v4loMBgM5R2Pd/pg3so1GAwGG8bpG8odRhjdcxkxYkSZDWlcVvBop9+xY0fWrVtXJiLbGTwPI4wOTzzxBO3btycoKAgfn2tXgB86dIgHH3yQRo0aERgYSK1atRg5ciRHjjiH4Dp58iSjRo2iRo0aBAYG0qxZM955553SOoxcSUpKYuDAgVSoUIEKFSowePDga6Kt5sTGjRvp2LEjAQEBVK9enalTp14TkXXlypV07tyZsLAwIiIi6NGjB7GxsSV1KHY82ulHRETQoUMHu4qQweCpuEsYPTMzk6FDh/Lwww/nmB8fH8/Fixd566232LFjBwsWLGDnzp1ER0c7OcERI0awefNmvvzyS3bt2sWECROYMGEC//vf/4rV3vT0dJfLZmVl0bt3bw4cOMDy5cv56aef2LNnD3fffXeeUoyHDx+mR48eNGnShC1btjB79mzmzJnD008/bS9z6NAhevfuTevWrdm8eTOrV68mLCyMnj17cvFiCcuOuhKgpzS34gq4pqqalpam8+bN07i4uGJr02Dheg64ZoTRi0cY3ZGPPvpIvb29XSq7ZcsWBXT79u32tLCwMH377bedyt100036xBNPFMiO4cOHa/fu3a/Zf/vttzUqKkpFRFNTU11qa9myZQro7t277Wk7duxQQFetWpVrvalTp2rNmjU1MzPTnjZjxgwNCgrSlJQUVVX95ptvFNDz58/by2zfvl2BPP1VcQRc8+iRPlhGEN999527zSgXnH7p23y3iz/GOZW/9KtFki/rwiWX6mcvX1gWLlzIqVOnWLNmDfPnz2fRokVMnTrVnm8TRl+/fj3r1q2jUaNGREdHc+rUKcAijOHt7c1bb71FUlKSXVfVJoweHh7OypUr+e2335gwYUKuwujr1q3jwoULjBo1KldbbcLo7du3Z+vWrWzdupXp06fnKYxusykpKYnFixfj4+NzjTD64MGD2b59O4sWLeLPP/+kf//+9hGso0B6cXP27FkAJ/v/9re/8dVXX3H8+HFUlZUrVxIfH18gJa/c2LRpEytXruTbb79l27Zt+Pn5MW/ePESEP//8M9d6NoF0R1nKFi1aUKtWLX799dc869lE1m1ER0eTmprKb7/9BljEdgIDA5k7dy7p6elcunSJDz74gAYNGhQ4LHVB8egwDH5+foSEhJgHuYZrMMLoRRdGLwwpKSlMnDiRe+65hwYNruoq/+9//2PkyJFUq1YNHx8fvLy8mD17NrfffnuR+/Ty8uLTTz91muYNCwujSZMmdv2AnMhJBB0sU2n5iadnV99yFE8HqF27NqtWrWLgwIFMnjyZrKwsGjduzE8//YS/v3+Bjq+geLTTBxNpszSJmNI3/0K5lPcKDSxQ/YKWz44RRi+6MHpBuXjxIn369MHHx+casZLp06ezb98+fvzxR2rUqEFMTAzjx4+natWq3HnnnUXqt1mzZtc81+vXr981F/bS5MSJE4wcOZK77rqLkSNHcuXKFV555RXuuOMONm/eTGhoaIn1bZy+wZADvXv3JjIykpkzZ1K7dm38/Pz429/+5lZh9Mcff5yffvqJ5cuX8+yzzzJjxoxcdWZdEUa/7777rqlXUnrS586d48477yQ9PZ2ff/6ZsLAwe15CQgJvvPEGGzZsoH17i/x2q1at2LZtG//+97+L7PSLIp7+888/X5N+/PjxIounz5gxA1V1uvAuWLCA8PBwPv/88xJdUuzxc/rG6RtywiaMbiMnYfQpU6bQs2dPmjdvTkBAQIGF0Ysbmyj6jz/+yOjRo5k7d26uZW3C6N99912ewujZt5JY6ZacnGx/nrB8+XIqVqzolG+TZMx+d+Tt7Z3nKpmSplOnThw4cIC9e/fa03bt2sXhw4f529/+lmc9m8i6jaVLlxIUFMSNN94IWO56sh+vl5cXXl5eJX7MLjl9EYkWkXgR2SciU3LInygiu0Rku4isEJEoh7xMEYmzbouL03hHNCOTs+8s5dL6PU7p77//Pp9//nlJdWu4TjHC6EUXRrfZFhcXx6FDhwCIi4sjLi6OlJQUwDKH3aVLF0SEjz/+mNTUVLs9trumZs2a0bhxYx599FHWrl3LgQMH+PDDD/nkk0/o379/oc9bXnzzzTc0bdr0mvcFHLntttu46aabuPfee9m0aRMbN27k/vvv569//Stdu3a1l+vevbvTIoCHHnqIc+fO8cADD7Bz504WL17Ms88+y/jx4+13HX369OGPP/5gypQpxMfH8/vvvzNixAhEhB49epTIMdvJb3kPFonEBKA+VzVym2crcwsQZP38EPC5Q16KK8uIbFtRlmwef+h9Pffp6kLXN7iOJyzZnDRpkkZERGhISIiOHj3aaSlfTEyMtmrVSv39/bVx48a6cOFCbdCggU6bNs1e5scff9SmTZuqr6+v05LNjRs3avfu3TUoKEhDQkK0ffv29uWetiWbjqxZs0YBPXDgQI72Hj16VPv166c1a9ZUPz8/rV69uo4ZM0bPnj1rL4PDks3hw4fnu2Rz9erV2r17dw0JCdGgoCBt2rSpPv7445qenq6qqqtWrcp3aaLtXObUl63eRx99lGN+9rYTEhJ00KBBWq1aNQ0ICNAmTZroK6+84rTs0bYUNS9yW7KZHZtduZ1zG0ePHtUBAwZoSEiIhoaG6sCBA/X48eNOZaKiopzOrarq+vXrtUOHDurv769Vq1bVKVOmaEZGhlOZhQsX6s0336yhoaEaHh6ut9xyi65ZsyZPe4pjyaZoPrcSItIBmK6qPa37U60Xi3/nUv5GYIaqdrLup6iqy/eMbdu21cK+lXZq2pd4hQcT/sQd9rSNGzeyatUqJk+e7DSvaSgaf/zxh3nT2VCq3H///Rw/fpxly5a52xS3kdf/nYhsUdW2+bXhyvROTeCww36iNS03RgM/OuwHiEisiGwQkbtzMXastUzsyZMnXTApZ7wjQ8k8ed4pbfXq1UydOrXk33IzGAwlRlZWFitWrCgToRmud4p19Y6I3Au0Bbo6JEep6hERqQ+sFJHfVTXBsZ6qzgXmgmWkX9j+vStXIG3HYcstjHVU7xh0zYRjMBiuT7y8vPKcfze4jisj/SNAbYf9WtY0J0TkNuBpoI+qptnSVfWI9e9+IAa4sQj25ol3ZChcySDr/CV7mom0aTAYDFdxxelvBhqJSD0R8QMGA06rcKzz+HOwOPwTDunhIuJv/RwJdAJ2FZfx2fGOtLzQkJl8dYrHOH2DwWC4Sr5OX1UzgEeBZcAfwBequlNEnheRPtZirwIhwJfZlmY2A2JFZBuwCnhJVUvO6VeuAEDmyQv2NOP0DQaD4Souzemr6hJgSba05xw+56haoKrrgL8UxUBX0SxF/CyHk+Uw0m/atCnHjh2zO3+DwWAoz3hMGAa9dIWL32/Du1pFcIip4uvrS9WqVd1omcFgMJQdPCYMgwT5gbcXIf3aE9yrtVPeiy++yKJFi9xkmcFgMJQdPMfpi+BVIcBp5Y6NWbNm8f3337vBKoPBYChbeIzTB/CqEEjajkOceuZz1CHYkQm6ZnDECKN7LkYYPX88zumTkYV31TA0LcOebpy+oSQo78Lov//+O/fddx9169YlICCAevXq8cQTT9iVsWzUrVsXEXHacopS+fvvv9OnTx8qVqxIUFAQrVq1yjfgW0lTGGH0nTt38n//9380atQILy+vXAcYmZmZvPTSSzRp0gR/f3+qVKnCQw89VBKH4YTHPMgFi9P3rlSBkAE34xV4NWZ5eHg4Bw4ccKNlBkPJUlJx8PNi69athISE8P7771O/fn327dvHI488Qnx8PD/++KNT2cmTJ/PEE0/Y97NrCmzbto3OnTszatQopk2bRkREBHv37iU8PLxYbU5PT89TLcsRmzC6l5cXy5cvR1V5+OGHufvuu1m7dm2usbxSU1OpU6cOffr04Y033si1/REjRrB+/XpeeeUVWrduzYULF/KUbyw2XInKVppbUaJspied0XMfrtb0xNOalZVlTx85cqTWqVOn0O0arsUTomwaYfTiE0a38dVXX6mI6Llz55zO1T//+c8863Xp0kUHDx5c5P7LgjC6I127dtXRo0dfk75y5Ur19vbWHTt2uNSODSOMng2vCoGoKqdfWsTFb69G6pw9e7YZ6ZcC3bp1u2abNWsWYBn95JRvmx5JTk7OMd+mhXD48OEc8wuLEUYvGWH0s2fP4ufnh4+P8yTCjBkzqFSpEi1atOCxxx6zn0ewfPerV6+mZcuW3HHHHVSuXJk2bdrkKRJTEEpbGN0VvvrqK+rXr8/PP/9Mw4YNqV27NgMHDrTrEpQkHjW9I4F+iK9ljb5jtM2SFho2XH8YYfTiF0Y/duwY06ZN49FHH3WqN378eG644QaqVq3K7t27eeaZZ1i2bBlxcXEEBgaSkGCJv/jiiy8ybdo0/vWvf7Fu3Toee+wxRIQHHnjAZRtyorSF0V0hISGBQ4cO8cknn/Dee+/h7+/P008/za233sqOHTty/B0UF57l9EXwCg3EK8ifzOSroRg2bNjABx98wEsvvUSlSpXcaKFnk9eoMCgoKM/8yMjIPPNr165d4FFnXhhh9OIVRj9x4gS33347rVq14t//dpba+H//7//ZP//lL3+hTZs2NGzYkG+++YahQ4fa74LuvPNOJk+eDEDr1q3ZtWsX77zzTpGdflkURs/KyiItLY1PPvmEFi1aAPD5559TvXp1lixZUmKKYeBhq3cAvCoEIH4+TiP9Q4cO8f77718jVmww5Ebv3r05dOgQM2fOZMOGDcTFxVGlShW3CqNv2bKFHj168Msvv9CyZUvmzJmTa3lXhNFt0oa2be/evfTq1avAx5OYmEjXrl2Jiori66+/zvdBaf369alatap9asUmFm5zfjZatGiR70XWFYoijJ6Tz8hPGN3VtkXESRClSpUqREZGFssx54UHOv1A8PYm6+xFNN0iWm2CrhmyY4TRi0cYPSEhgc6dO9O8eXO+/vprl6ZSjxw5wokTJ6hd2xKxPSoqitq1a19zZxEfH0/dunULZE9xUlhhdFfo3LkzqsqePVc1vU+dOkVycnKJH7NHOn3x9wWFzFOWKR7j9A3ZMcLoRRdG37VrF507d6ZJkya8/fbbnDp1yt6X7WK4fv16XnvtNbZu3crBgwdZtmwZvXv3pk6dOvbpFRFh6tSpLFy4kJkzZ5KQkMAnn3zC3LlznZZ5FiclKYx+5coVJ4H406dPExcXx65dVwMMDxkyhHr16jFq1ChiY2PZvn079957Lw0bNizU3VaBcGWJT2luRVmyqaqannRWT7/xox4bMUsv7zikqqoHDhxQQD/88MMitW24iics2TTC6EUTRs9taajjsWzZskU7dOig4eHh6ufnp/Xr19dx48ZpUlLSNe29/fbbWr9+fQ0ICNAWLVroe++9l2N/eVEWhNFt/ib7FhUV5VQvISFB77rrLg0ODtbKlSvrgAED9NChQ3naUyrC6KVNYYXRVRVNz0Qvp3Nh/jrSNu0hdHhXgro15/z581SrVo3XX3+9VN54Kw8YYXRDaWOE0YtHGN2l1TsiEg38B/AG3lfVl7LlTwTGABnASWCUqh605g0HnrEWfUFVP3alzwKTnsmV+CS8q1dEgv3BS+wPcytUqEBqamqJdGswGEoemzD6qlWr3G3KdU++Tl9EvIGZQA8gEdgsIovVWQHrN6CtqqaKyEPAK8AgEYkApmERS1dgi7XumeI+EHy9wUsgLR3vsCB8G9XAt05ksXdjMBhKHyOMXny48iC3HbBPVfer6hVgAdDXsYCqrlJV21B6AxbxdICewHJVPW119MuB6OIx3RkRQQJ8ybqcjldoAH6NahDQ/mo0wylTpjitVTYYDIbyiCtOvyZw2GE/0ZqWG6MBW7SlgtYtEl4BfujldLwqBJKVcpnMlMv2vNWrV/Pzzz+XVNcGg8FwXVCsSzZF5F4sUzmvFrDeWBGJFZHYkydPFr7/AF/IUryC/Uk/eILk8R+haemACa9cEpS1RQAGgydTXP9vrjj9I0Bth/1a1jQnROQ24Gmgj6qmFaSuqs5V1baq2tbx9fSC4hVgfRPQzwfviiEERbdGsywnyjj94sXX19e+pttgMJQ8ly5dcjksdF644vQ3A41EpJ6I+AGDgcWOBUTkRmAOFofv+NriMuB2EQkXkXDgdmtaiSBWpy++3niFBuLfKsoeV984/eKlSpUqHDlyhNTUVDPiNxhKEFUlNTWVI0eOUKVKlSK3l+/qHVXNEJFHsThrb+BDVd0pIs9jeRlgMZbpnBDgS2uMj0Oq2kdVT4vIP7FcOACeV9US87zi7YX4epOlWai3kJF4Cu+qFfCOCKFWrVpUrlyZzMxMp0BbhsJRoUIFAI4ePUp6erqbrTEYPBtfX1+qVq1q/78rCh7zcpaNK3+eRK9kcGXXUVK/jyWoe0tCh3QqRgsNBoOh7OHqy1keF3tHAvzQtAy8KwQigX5OIZYNBoOhvONxTt/2MFeC/BE/X/tbuXFxcdx2221s377dneYZDAaDW/EYp3/p0iXW/LqW5BRLJEEJ8EECfMlMtjj9tLQ0VqxYQWJiojvNNBgMBrfiMcpZvr6+7N27j6CgIML8IsHXxzLVcymdrItpJryywWAw4EEjfR8fHypWrEjyyWTE3xfx9rJP9WSePG+cvsFgMOBBTh+gcmQkyaeSLTF4sjKRYIu4cGbyeSpWrAgYp28wGMo3HuX0IyMrcflyGumShWQq3pUta1ozT17A29ubv/71r3bnbzAYDOURj5nTB4isbAmlfPbSRSrhjVfFIPDxtj/MLYpkncFgMHgCHjXSjwgPx8vLi+NnLVM4EuCLb6MaBHRo7GbLDAaDoWzgUU7f29ubiIhwjiWfAG8vxM8Hn8ph+FQJA2DixIkMGTLEzVYaDAaD+/Aopw8QGRlJcvIpS/A1H280LZ3LsQmoKklJSWzZssXdJhoMBoPb8DinXzkykvT0dNK9FLwsK3cufLqGrPOXTKRNg8FQ7vGoB7lgGekDnE9LJdzbG+9qFfFrWRuvIH8iIiI4c+YMWVlZeHl53PXOYDAY8sXjPF/FimH4+Phw8twZRATviBC8gvwRX28iIiLIysri/Pnz7jbTYDAY3ILHOX0vLy8qRUSQePK4JSHAlyt/JHJlTxKNGzemR48eXLlyxb1GGgwGg5vwuOkdsKzX3707Hhpagq6l7TyMhPpz56g7ufPOO91tnsFgMLgNl0b6IhItIvEisk9EpuSQ30VEtopIhogMyJaXKSJx1m1x9rolQeXISDIzM8nwUsTH2xJt8/i50ujaYDAYyjT5On0R8QZmAr2A5sAQEWmerdghYATw3xyauKSqra1bnyLa6xKRkZUASEm/DD5eeAX4kXnyPAcOHKB+/fp8/fXXpWGGwWAwlDlcGem3A/ap6n5VvQIsAPo6FlDVP1V1O5BVAjYWmAoVKuDn50tyyjkkwNcSgO1cKoEBARw4cIBjx46520SDwWBwC644/ZrAYYf9RGuaqwSISKyIbBCRu3MqICJjrWViT548WYCmc0ZEqFQpkqMnT4CPFxLkD1lKmPoBJtKmwWAov5TG6p0oq1jvUOAtEWmQvYCqzlXVtqratnLlysXSaeXIShw5edwSVz8sCADv85cJCQkxTt9gMJRbXHH6R4DaDvu1rGkuoapHrH/3AzHAjQWwr9BEVo4kKyuLTG/wCg8GIDP5gnkr12AwlGtccfqbgUYiUk9E/IDBgEurcEQkXET8rZ8jgU7ArsIaWxBsb+ZezLiCVLCM9DNOnKNv3760bt26NEwwGAyGMke+6/RVNUNEHgWWAd7Ah6q6U0SeB2JVdbGI3Ax8A4QDd4nIP1S1BdAMmCMiWVguMC+paqk4/ZDgYAICAjh98QK1A30Rf18yj53j7bffLo3uDQaDoUzi0stZqroEWJIt7TmHz5uxTPtkr7cO+EsRbSwUIkJkZCWSTp+kTlgUfn+pS2CPlu4wxWAwGMoMHheGwZHKkZEknjyOBPjiFeQP6Zk8+eST1KtXz92mGQwGg1vwaKcfGRnJlYx0svy8yEq9zKWff8dbvDh69Ciq6m7zDAaDodTxcKdveTP3smagaRmkxR0gPDCEK1eukJqa6mbrDAaDofTxyIBrNoKCgggODuLspYtUqRuJd61KRF74HYBTp04RHBzsZgsNBoOhdPHokT5AZKVIjp85ZXkrNzWN8IoVAfNWrsFgKJ94vtOvHEnS6ZOIvy9X9iVR+5QwduxYQkND3W2awWAwlDoePb0DlhU8v13+Dfx9yDp1gYbhVZkzZ467zTIYDAa34Pkj/chKZKmS4S+WuPrJF8jKyiI9Pd3dphkMBkOp4/FO39/fn9DQUM5lpiGB/lxMPoOPjw+vv/66u00zGAyGUsfjnT5YpnhOnD+DV0gAARmCn58fZ86ccbdZBoPBUOqUC6cfGVmJk+fO4FUhEICIsIpm9Y7BYCiXlA+nXzmSM6kpiDXEcnhoBeP0DQZDuaRcOP1KlSpxKT0NCbeEWLTJ6REAAB9TSURBVK7ob4RUDAZD+cTjl2wC+Pn6UrFiRS4F+4CXMPCmbvh3auxuswwGg6HUKRdOHyzyiafTL1IpOID7W/UgfFwvd5tkMBgMpY5L0zsiEi0i8SKyT0Sm5JDfRUS2ikiGiAzIljdcRPZat+HFZXhBiYyM5HjKeQLaNiSgbUP279/Pf/7zH3eZYzAYDG4hX6cvIt7ATKAX0BwYIiLNsxU7BIwA/putbgQwDWgPtAOmiUh40c0uOJGRkZy9nIL4+5J1LpU5c+bwxBNP8PXXX7vDHIPBYHALroz02wH7VHW/ql4BFgB9HQuo6p+quh3Iyla3J7BcVU+r6hlgORBdDHYXmIiIcM5fTiXz/EVSV2znH1Oe4eabb2bUqFHs37/fHSYZDAZDqeOK068JHHbYT7SmuUJR6hYrPj4+VKhYgSxvgYxMfK5k8cUXXyAiDBw4kLS0NHeYZTAYDKVKmViyKSJjRSRWRGJPnjxZYv1ERkaSWi0Y/9YNkABf6taty7x589iyZQuvvvpqifVrMBgMZQVXnP4RoLbDfi1rmiu4VFdV56pqW1VtW7lyZRebLjiRkZGczrgEQNY5i3JW3759WbBgARMmTCixfg0Gg6Gs4IrT3ww0EpF6IuIHDAYWu9j+MuB2EQm3PsC93ZrmFipHViI5PZW0bQdIWbzFnj5o0CCCg4NJSUkhMTHRXeYZDAZDiZOv01fVDOBRLM76D+ALVd0pIs+LSB8AEblZRBKB/wPmiMhOa93TwD+xXDg2A89b09xCeHg4Z65cRDMzyTx21ilPVYmOjqZPnz5cvnzZTRYaDAZDyeLSnL6qLlHVxqraQFVftKY9p6qLrZ83q2otVQ1W1Uqq2sKh7oeq2tC6fVQyh+EaXl5e+IYEIsEBZJ29iGapPU9EmDJlCr/99hsTJ050o5UGg8FQcpSJB7mlSeXKkVClAnoxjXNzf0Yzr64y7d27N08++SSzZ8/m888/d6OVBoPBUDKUO6cfWakSZ5uE41O3Cmkb93Fu5jI0I9Oe/+KLL9KxY0fGjBnDnj173GipwWAwFD/lzulXrhzJptN/4tujJT4NqpH225+ceWsJmm5x/L6+vixYsIDOnTvj7+/vZmsNBoOheCl3Tj8sLIxMzWKnnCG4Xzt8G9cgfWciZ15djKZZdHNr167NkiVLiIqKQlXzadFg+P/tnXuUXFXV4H/73np2VXX1uzv9zJuQECAkJIAfb4LxQ0FdmQg6osgQcXT5wCUOw4jKOAzOclRYvgaNzwVGgSDRD8Uo+Bk+HiFAEmgS8iDppLvTr+rqrqqu971n/rjVRXdISAMJTXWf31p31X2cuvfsOt1737PPPmdrNKXDtFP6IkJ1TTW9fX14Tp1B8JrzcJ/aQm5/P5kdh8aVjcfjfPCDH+SLX/wi8Xh8kmqs0Wg0J45pp/QB2lpbiUQG+eumv2HVhwh94nz8Fywi295FZmc3KpcHwO/309jYyN13382CBQtYv369fvPXaDQlzbRU+osWLeQ9551LT08Pf3h4IxGVJnDVWUjIR+L+pxm49XfYyQwul4sf//jHPPXUU8yYMYNrrrmGSy+9lK6uiU5I1mg0mncX01LpiwgLFpzC+99/BS6XySN//gvtB/ZSdsUZmA0ViGmS2vIqtu2Ec65YsYJnnnmGH/3oRwwMDBAOhydZAo1Go3lryLvNXbFs2TK1devWd+x52WyWzU/8BwcOdNDS0sz5552H/cQ+rO4hJOzD9555uOsriuVt28YwDDKZDFdccQWf/vSnWb16NSLyjtVZo9FojkREnlNKLTteuWn5pj8Wj8fDJRdfxLnnrKCrq5uH//RvjJxZhzmnltTjLxG9fQPpFw8WffmG4fxkvb29RCIR1qxZw3vf+15eeeWVyRRDo9FoJsS0V/rguHsWLjyV97//XxER/vTIX9hfZVH2viWoXJ7h7/4bkdvuJ739QFH5t7a2snXrVn7wgx+wZcsWFi9ezE033UQ2m51kaTQajebYTHv3zpFkMhn+ufkJDh48RFtrK+e2nEJm4wvkdndD3sJsribwoWX4zpyFGI5Lp7e3l69+9au0t7ezZcsWRIQnnniCJUuWEAgEJk0WjUYzfZioe0cr/aOglKK9/WW2PLuVQCDA2cuW0ihlpDZsJburE/I2ZlMVgauW4TtrFmI6HaZcLofb7SYej9PQ0IBpmlx99dVcf/31LF++XPv9NRrNSUMr/RNAX18f//7PzcRicfx+H/PnzWdeeR32n3aQ3XkIMU38H1iGd1Ez7uZKxGUCjtHYvHkz69at4/777yeVSrFo0SJ++MMfcuGFF06yVBqNZiqilf4JQilFZ2cXu3a9wqFCgpWW5mYW1jQT2jMEI1mUsskfGsD/vjPwLmzGCHiLb/WxWIz169ezbt06fvazn7F48WK2b99Oe3s7l112GXV1dZMpnkajmSJopX8SiCcSvLJrN6/s3k06nSYUDLJo5lyaOtJkHt2OZ1ErrsYq8JoYFWV4z5qFGfK9zq2zdu1afvrTnwJw5plncvnll7Ny5UouvfRS7QLSaDRviROq9EVkFXAXYAI/U0rdecR1L/BrYCkQAT6ilDogIjNxsm2NxjM+rZS68Y2e9W5W+qNYlkVHx0F27tpFT08vhmGwoK6FOYFa/IMZss/tI98VQXweXDNr8S6bjWfpLFzlZYghWJbF888/z6ZNm/jrX//Kk08+yYwZMzhw4AAiwiOPPEJLSwunnXaaNgIajWZCnDClLyImsBtYCXTipD28Rin18pgy/xU4XSl1o4hcDXxIKfWRgtL/k1LqtIlWvBSU/lii0SF2vfIKe/bsJZfL4XK5mBWqYXYfeF8dxO4fBgXic2O21uA9axbe5XNxlfuLA8CJRIL9+/ezePFilFI0NjbS09NDQ0MDF198MStWrOCSSy5h8eLFkyytRqN5t3Iilf65wDeUUu8tHN8CoJT632PKPFoo85SIuIAeoBZoY4or/VFyuRzd3Yfp6u6mu6ub4VgMgGrTz6K4n3BnAtUXA6UQrxvXzFp8552CWR/GbAhjBn1FI9DZ2VnsBWzevJmuri4+//nPc9ddd5HL5fjyl7/MsmXLWL58OfPnzy9OGNNoNNOXE6n0VwOrlFL/pXD8cWCFUupzY8q8VCjTWTjeB6wAgkA7Tk8hBvwPpdTmozxjLbAWoLW1dWlHR8eEhHw3k0gk6Orupqurm+7uw2QyGXx5WJwIUNefw+X14GmrRylF5vm9uGc34DmjDaM2hKuxAldFoBgN1NXVhVKK5uZm9uzZw9KlS4tLPYfDYc4++2xuvfVWLrroInK5HEopPB7PZIqv0WjeYSaq9F0nuR6HgValVERElgJ/EJFFSqnY2EJKqXuAe8B50z/JdXpHCAaDnDJ/PqfMn49SikhksGgEnq/uxbKy+FWaVgkxO+zDNAXr4CC5nd3EduzHrApizqjCPbeemvmNuGaEUbbNvHnziEaj7Nq1iy1btrBlyxaeeeYZLMvJ/PW3v/2NK6+8knnz5rFo0aLitnLlSioqKo5Ta41GM9WZiNLvAlrGHDcXzh2tTGfBvRMGIsrpRmQAlFLPFXoA84HS89+8DUSEmppqamqqOeP0xViWxeDgIP39A/T3D/BkucXwcC8u1U+zuJlXF8A7mMLq3U92237nHmVejHAZRk0IV1MVcxY0ccpV/4lPfvzaYo8AoK2tjZtvvpn29na2bdvGgw8+iFKKHTt2UFFRwYYNG/jtb3/L3LlzmTNnDnPmzGHu3Lk0NTVpN5FGMw2YiNJ/FpgnIrNwlPvVwEePKLMR+ATwFLAaeEwppUSkFhhUSlkiMhuYB7x6wmpfopimSW1tLbW1tcVzmUyGgYEI/QMDvNLfT3//AAwlmRE3qE8ahJPgiiawDkfJvXgQuyeGUV6GlUhhx5N4z5qNWVvOnIpqbr/5VoyQDzEMUqkUu3btYsGCBQBEIhF27NjBww8/TC6XKz4/Go1SUVHBvffey9atW4tGYebMmbS0tOjlJDSaKcJxlb5SKi8inwMexQnZ/LlSql1Ebge2KqU2AuuA34jIXmAQxzAAXADcLiI5wAZuVEoNngxBSh2v10tTUyNNTY2AMyksmUwyEIkwFB1iT3SI6FCUkb4hKhM27lCcOi/U9SYxX+3DrK/EPjxMdm839kAMCfgwQn6MqgBz6sJkR16FmTVc/4lPcsMNN2BZFocOHWLv3r10dHQUXT/btm3jnnvuIZlMFutWXl7O8PAwAN///vfZv38/bW1ttLa20tbWRltbm55kptGUCHpyVolh2zaxWIzo0BDR6BBD0SGig4PE4nFCuJk9YFAVzeNN5jGSWchZ474vfg8S8uM75xSMMg92KoMR8uGe24BZUYYUJpP19PSwb98+Ojo6GBkZYe3atQBce+21bNiwgZGRkeI9Fy1axEsvvQTAZz/7Wfr7+2lsbKSxsZGmpibmz5/P2Wef/c79SBrNNETPyJ1mWJZFIpFgOBYjFosTj8UYjsVIDgzjGhihPuOiMmNSlrRwKQPPGTMx84rsc/tA2XjPnANAduchFAoj6MeoKMOsDmLWhnE1VWA2VGKEnfkFQ0NDHDx4kNFIqyuvvBKAj3/84zz77LN0d3cXI4xWrVrFn//8ZwBOP/100uk0DQ0N1NfX09DQwPnnn8+aNWsA2L59O9XV1dTV1ekIJI3mTfBuid7RvEOYpkk4HD5qKkfbtonHE8RiMYbjceLxOCOJERKJBMZCk2DejdcfJ2z4qHSDGc9iDyZgvz3+RiLOJLPaMJ7Frczxe5k5GMasryCzsxsj6OOXP/h/SMCL4TKJx+N0d3ePSya/atUqDh48SE9PDy+++CKbNm0inU6zZs0abNtm2bJl5PNOYvqqqipqa2tZu3YtN910E/l8nm9+85vF8ZDRra2tTUcmaTQTRCv9aYBhGITD5YTD5Ue9ns/nSSQSJEZGiJ3RzEgiQSKeIBWNYQymCI1YlKeFsrTCk7awfCZYFuZQgszmnbgaq7B7YijLJv30LsTjQnwexO+hJuDFCPkZfrYfszrIN1avxVVfgVkRQAIeDJdZzEWslOKBBx6gt7eXnp4eent76e/vp7q6GnAGm++4445i+VG+9a1vceutt9LZ2cmFF15ITU3NuG3NmjWsWLGCeDzOtm3bqK6uprq6mqqqKtxu98n98TWadxla6WtwuVxUVFQc821ZKUU2m2UkmSQ5kiSeTBb2R8hf0ebkFzCT+DKKmpYwnoyFkc4j0TgcjsIRLkRXay3umfWoXJ7sywdxzW7AbK4CpbhwIIARPg2j7RzM6hCumiBSFUTlbWpra8lms0SjUfr7++nr66O/v5+FCxcW733OOecwMDBAT08PL730EgMDAyxevJgVK1bQ3t7OBRdcMK4u5eXl/OY3v+HKK69kx44dfPvb3y4ahcrKSiorK1m5ciUNDQ0kEk5vqbKyEp/v9QvpaTSlgFb6muMiIni9XrxeL1WVlccsZ9s2mUyGVCpNKpVytmSSdDSOPZjAO5TBE8+hQj6MUB5/2sZnivNXmMuj+mLknj9KRK8hiNsFbtNZwmLeDBobKmi0y7CjVXgGDDIvd1HrMvnF//weRnVw3LIWo+6lBQsW8OijjxKJRBgcHCQSiRCJRJg5cyYAAwMDPP3000QikWK0EsA//vEPGhoa2LhxIx/72McAJ7fyqFF48MEHWbhwIY899hj33Xcf4XCY8vLy4ufq1aspLy+nr6+PaDRadMNpw6GZDLTS15wwDMPA7/fj9/uBYxsHcBRxJpMhdUWadDpNIp0mlSgnd0YYFU3hHk7jTuRwpfK4UhZm1sLM5JGMhW1ZGMks9uEo2Rc7kFQOK1RG/vAguT3dzgNcBuJygceFeF3gdYHPw1l+N+bCUzHnBTFshZGzcdc3YKeyXHTBhezbtw9wXF5DQ0MMDQ3R2OiE0S5fvpyf/OQnRKPRcVsoFAKgo6ODRx55hFgsNi666bLLLqO8vJx77rmHr33ta8Xzbreb8vJy9uzZQ2VlJevWreOPf/xj0SiMGo6bbroJ0zR5+eWXiUQixWvl5eWEQiHtotK8KXT0jqYkUEqRy+fJpNNkMhnSmQyZRIp8JIYlgpGzcfUncHfHMFN5zHQeyVpIJo9k86isBbk8WDbepXMxAj7yXQPk9vXgO3cB4naR299Dvivi9CgKPQs8LvA5BkP8biTgx7NsFp7KIORtDMDdVou4xs9mzufzxONxYrEYTU1NuFwudu7cyQsvvMDw8PC47a677sLtdvO9732PX/7ylwwNDTE8PEw8HkdEyOVyiAif+tSn+MUvfjHuOaFQiFhhcb8vfOELPPbYY4RCIYLBIMFgkObmZu6++24Afve739Hd3V28FgwGqa2t5ZxzzgGcTHEul4tgMKgjp0oQHbKp0eAYC8uyyGQyZNIZsokU+UwGO5WHgThGbxxqQs5bf+cQ0htDsk6PglwelRvzaTkDyL73LERMg+y+w1g9UfzvWYgSyO7qxB6MF4yGiXI7PQ28LsTnRvweCPrwLJ+N6fdAPI3b48Ezp94xMNm8813jNbdUMpkszobes2cPHR0dThTW8DCxWAzbtvnSl74EwHe/+12eeOIJ4vG4MzCfSFBTU8Pjjz8OwCWXXFLcH+WMM85g27ZtAKxYsYItW7YATi8kFApxwQUX8NBDDwFwww03MDAwQDAYJBQKEQqFOP3004surw0bNqCUoqysjEAgQCAQoK6ujpYWZxWXdDqN1+vVLq2ThFb6Gs3bRClFLpcjm0qTTaXJJVJYgyMoBLJ5pDcGwymM+jCGBRzoRwaTTo8iZ0HOQo01GJYNHhf+c5wlMTLtHah0Dt/SuSgg88I+VDwFpgEuE9yFT88YA1Lph7NmIl4XRmcUCfnxnNqEx+9FhpKYZT6MMo8TPeU2x8mTy+UYGXFCdUc/DcNgyZIlgKO0Ozs7xxmN1tZWvvKVrwCwevVqdu/eTSKRKJZZtWpV0SjU1dXR398/7pkf/ehHuffeewEIBAKk0+miUQgGg1x77bXcdttt2LbNhz/8YQKBAGVlZcXt4osv5vLLLyebzXLffffh9/vHXZ85cyYzZszAsixGRkYoKyvD5ZqeXmsdp6/RvE1EBI/H47g6jhHu+kZYlkUunSGXyZJPZcjFU6hElhQ2ZC2odEM6R6YyiFg2xrwG1EgaydqO4chbqJyFSltIIgt5C4km8QaduqS37gG/Fw7GyAKpp3aOn4EtAq6C4Rj9rAthLmqi3GUQbu9C1ZXTk3wJ3CbvydRgtDVhhnyYAR+uoB9XyE8+Esfwurn/t+vBZY57Ux8bPvvMM88UDUoymWRkZISGhobi9dtuu614fXRrbm4GIJvNsn///uL5VCpFMpnENE0uv/xyhoeHue666173G99xxx3ccsstHDx4kNmzZwPOIPuo8bjjjju49tpr2bdvHzfeeGNxzGl0u+6661i+fDmdnZ2sX7++eN7r9eLz+TjvvPOYMWMGQ0ND7N27F5/PV7zm8/morKwsOSNTWrXVaEoI0zQxA2X4AmVv6ftKKSfc1VJYuTy5dJp8MoOdymGlsxizK1HZPFmvyxlfUHMhnXN6ITnbcRcVehyjBsTI2rijObBtUnv6cA9ncVsulGWR/o+d2DiLZOWOUSejpRo5pRFLKXhyN/bcOtTMGlQ2j2/HIXwekxr3GJdW7xB9u/8d0+9h7emXYbRU42qoAASGRnDVlWNZFh63h20vvFB0bY37DXAm6r366qtFYzC6zZ07F4CKigq+853vjDM4yWSy6FrK5/Mkk0kikchrkWWpFJdeeinLly9nz549xR7NWDZu3MgHPvABNm/eXJx1Ppa///3vXHLJJTzwwAPccMMNxZ7IqPH41a9+xamnnsqmTZtYt25d0WCMRsPdfPPN1NbWsnXrVp588kk+85nPnPSBee3e0WimOI7xAGwbZSuUZWFl8+QzWax0Fjudw8rkUAMJVDIL6Swqk3eMR7ZgNLIWkrcwwgFcNeWOsXmxA7O+ArM6hJ1Ik32xAywbZdlg28UxkLG45zTgaqrBHkmTeW4v7lNbMOrC5KNxrB0dYAjKEDAMMJ19ZQjKNFCGMDKvimx1GUYyR9n+KMkFtVhhH+54Bm9XzBkbcbsQt4l4TPC4MbyFY7cL1VoFZR4kmUWG01AfQtwu7GSaZCxOJpcjncuRzmVIZ7M0tTRRXl5Ob18fO7ZvJ5VOk0lnyGTSpNMZVl6+kob6Bna8uIOHNjxEOp0mlU6RSTthy1//+jdoaWnhL3/5C3ff7WS+y2azzhhTJkN7ezuzZs3izjvv5JZbbimOe7wVtE9fo9GcFMb2QJRtg63AtrHzNvlMDiuTxcrksNNZVCqLncwhqSwqm3cUsMcNqRyqO4pUBTF9XlQsSb5jwOmR5C3HcBS2sfuumXWYFUGsoQTZXZ14F7VhhPzke6Lkdh+Z5uP1mEtmQbgM6/AgvHIY69y5WD4X0jGA+9WBo8sLKAMQYe/SatI+obJzhLquNNvPDmObwoxDaSojWWyDcUbKMVwGGAKG0Le4BsM0CPWkKEvkSJ9aj9d0Iwcj5KJxwuecQuv5S95Su2ifvkajOSmIiDNeYIDw2mCxwVtXKEVDYiuUrRwlP7pvj+47n3bBZRXInYWdt8Gy8OTbsFcuRuXykHEGz+1sDvIW5GzHmFg2RsDJM2F73FjhIGZFCBHBbgDL63eMjHrt+dh2QesrlFKc7qpFMLH8MaxwjAukEVFCXiJYEnMMYd6RA6VA2Sg7X5StuTWAKMjuH8SKxPFX1gM5Mjuj2INx3M3J4/1Ubxut9DUazaQz3pCceIourrHGZey54vHYcqDs0Z6GKhqO0d6HyttIwRh5lrShbBuxXis/tofiRGQZjpvJZeJZ0gqG4axT5TbxnTcPlI1REzoJ0o9nQkpfRFYBd+EkUfmZUurOI657gV8DS4EI8BGl1IHCtVuA6wEL+LxS6tETVnuNRqOZAI5RgbEmZbrOFjhuUlQRMYEfAu8DFgLXiMjCI4pdD0SVUnOB7wHfLnx3IU4WrUXAKuBHhftpNBqNZhKYSCbs5cBepdSrSqkssB646ogyVwG/Kuw/AFwqTjDvVcB6pVRGKbUf2Fu4n0aj0WgmgYko/Sbg0JjjzsK5o5ZRSuWBYaB6gt/VaDQazTvERJT+SUdE1orIVhHZeuQ0bo1Go9GcOCai9LuAljHHzYVzRy0jIi4gjDOgO5HvopS6Rym1TCm1rLa2duK112g0Gs2bYiJK/1lgnojMEhEPzsDsxiPKbAQ+UdhfDTymnFlfG4GrRcQrIrOAecCWE1N1jUaj0bxZjhuyqZTKi8jngEdxQjZ/rpRqF5Hbga1KqY3AOuA3IrIXGMQxDBTK/R54GcgDn1VKWUd9kEaj0WhOOnoZBo1Go5kClOzaOyLSD3S8jVvUAEdfRKM0mWrywNSTaarJA1NPpqkmD7xepjal1HEHRd91Sv/tIiJbJ2LtSoWpJg9MPZmmmjww9WSaavLAW5fpXRGyqdFoNJp3Bq30NRqNZhoxFZX+PZNdgRPMVJMHpp5MU00emHoyTTV54C3KNOV8+hqNRqM5NlPxTV+j0Wg0x0ArfY1Go5lGTBmlLyKrROQVEdkrIv9tsutzIhCRAyLyoohsE5GSm7EmIj8XkT4ReWnMuSoR2SQiewqflZNZxzfLMWT6hoh0Fdppm4j862TW8c0gIi0i8riIvCwi7SLyhcL5kmynN5CnlNvIJyJbRGR7QaZvFs7PEpFnCjrvd4Vlco5/v6ng0y8kZtkNrMRZvvlZ4Bql1MuTWrG3iYgcAJYppUpyUomIXAAkgF8rpU4rnPs/wKBS6s6Cca5USn11Muv5ZjiGTN8AEkqp70xm3d4KIjIDmKGUel5EQsBzwAeBT1KC7fQG8qyhdNtIgIBSKiEibuAJ4AvATcAGpdR6EfkJsF0p9ePj3W+qvOlPJNGL5h1GKfVPnLWYxjI24c6vcP4hS4ZjyFSyKKUOK6WeL+zHgZ04OS9Ksp3eQJ6SRTkkCofuwqaAS3CSVsGbaKOpovSnarIWBfxVRJ4TkbWTXZkTRL1S6nBhvweon8zKnEA+JyI7Cu6fknCFHImIzASWAM8wBdrpCHmghNtIREwR2Qb0AZuAfcBQIWkVvAmdN1WU/lTlX5RSZ+HkJ/5swbUwZSgsv136/kX4MTAHOBM4DPzfya3Om0dEgsCDwBeVUrGx10qxnY4iT0m3kVLKUkqdiZOTZDmw4K3ea6oo/Qklayk1lFJdhc8+4CGmRn7h3oLfddT/2jfJ9XnbKKV6C/+UNvBTSqydCn7iB4F7lVIbCqdLtp2OJk+pt9EoSqkh4HHgXKCikLQK3oTOmypKfyKJXkoKEQkUBqIQkQBwOfDSG3+rJBibcOcTwMOTWJcTwqhyLPAhSqidCoOE64CdSqnvjrlUku10LHlKvI1qRaSisO/HCVjZiaP8VxeKTbiNpkT0DkAhBOv7vJbo5X9NcpXeFiIyG+ftHpxkN/eVmkwi8lvgIpwlYHuBrwN/AH4PtOIsob1GKVUyA6PHkOkiHLeBAg4Anx7jD39XIyL/AmwGXgTswun/juMHL7l2egN5rqF02+h0nIFaE+dF/fdKqdsLOmI9UAW8APxnpVTmuPebKkpfo9FoNMdnqrh3NBqNRjMBtNLXaDSaaYRW+hqNRjON0Epfo9FophFa6Ws0Gs00Qit9jUajmUZopa/RaDTTiP8PKHxy3pqkw8UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# 绘制每个回合的损失\n",
    "plt.plot(runner1.train_epoch_losses, label='batch size: 16, lr: 0.01', c='#9c9d9f')\n",
    "plt.plot(runner2.train_epoch_losses, label='batch size: 32, lr: 0.02', c='#f7d2e2')\n",
    "plt.plot(runner3.train_epoch_losses, label='batch size: 64, lr: 0.04', c='#f19ec2')\n",
    "plt.plot(runner4.train_epoch_losses, label='batch size: 128, lr: 0.08', c='#e86096', linestyle='-.')\n",
    "plt.plot(runner5.train_epoch_losses, label='batch size: 256, lr: 0.16', c='#000000', linestyle='--')\n",
    "plt.legend(fontsize='x-large')\n",
    "plt.title('epoch loss with different bs and lr')\n",
    "plt.savefig('opt-mnist-loss.pdf')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，如果按每个回合的损失来看，每批次样本数越小，下降效果越明显。适当小的批大小可以导致更快的收敛。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**动手练习7.1**\n",
    "* 尝试画出按迭代的损失变化图，观察不同的批大小在两种损失图中的下降效果是否相同。\n",
    "\n",
    "**动手练习7.2**\n",
    "* 对比下面两种实验设置，分析和比较它们的区别。1. 批大小增大一倍，学习率不变。2. 批大小不变，学习率减少一半。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 7.3 不同优化算法的比较分析\n",
    "\n",
    "除了批大小对模型收敛速度的影响外，学习率和梯度估计也是影响神经网络优化的重要因素。神经网络优化中常用的优化方法也主要是如下两方面的改进，包括：\n",
    "\n",
    "* 学习率调整：主要通过自适应地调整学习率使得优化更稳定。这类算法主要有AdaGrad、RMSprop、AdaDelta算法等。\n",
    "* 梯度估计修正：主要通过修正每次迭代时估计的梯度方向来加快收敛速度。这类算法主要有动量法、Nesterov加速梯度方法等。\n",
    "\n",
    "除上述方法外，本节还会介绍综合学习率调整和梯度估计修正的优化算法，如Adam算法。\n",
    "\n",
    "### 7.3.1 优化算法的实验设定\n",
    "\n",
    "为了更好地对比不同的优化算法，我们准备两个实验：第一个是2D可视化实验。第二个是简单拟合实验。\n",
    "\n",
    "首先介绍下这两个实验的任务设定。\n",
    "\n",
    "#### 7.3.1.1 2D可视化实验\n",
    "\n",
    "为了更好地展示不同优化算法的能力对比，我们选择一个二维空间中的凸函数，然后用不同的优化算法来寻找最优解，并可视化梯度下降过程的轨迹。\n",
    "\n",
    "**被优化函数** 选择Sphere函数作为被优化函数，并对比它们的优化效果。Sphere函数的定义为\n",
    "$$\n",
    "\\mathrm{sphere}(\\bm x) = \\sum_{d=1}^{D} x_d^2 = \\bm x^2,\n",
    "$$\n",
    "其中$\\bm x \\in \\mathbb{R}^D$，$\\bm x^2$表示逐元素平方。Sphere函数有全局的最优点$\\bm x^*=0$。\n",
    "\n",
    "这里为了展示方便，我们使用二维的输入并略微修改Sphere函数，定义$\\mathrm{sphere}(\\bm x) = \\bm w^\\top \\bm x^2$，并根据梯度下降公式计算对$\\bm x$的偏导\n",
    "$$\n",
    "\\frac{\\partial \\mathrm{sphere}(\\bm x)}{\\partial \\bm x} = 2 \\bm w \\odot \\bm x,\n",
    "$$\n",
    "其中$\\odot$表示逐元素积。\n",
    "\n",
    "将被优化函数实现为OptimizedFunction算子，其forward方法是Sphere函数的前向计算，backward方法则计算被优化函数对$\\bm x$的偏导。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nndl.op import Op\n",
    "\n",
    "class OptimizedFunction(Op):\n",
    "    def __init__(self, w):\n",
    "        super(OptimizedFunction, self).__init__()\n",
    "        self.w = w\n",
    "        self.params = {'x': 0}\n",
    "        self.grads = {'x': 0}\n",
    "\n",
    "    def forward(self, x):\n",
    "        self.params['x'] = x\n",
    "        return paddle.matmul(self.w.T, paddle.square(self.params['x']))\n",
    "\n",
    "    def backward(self):\n",
    "        self.grads['x'] = 2 * paddle.multiply(self.w.T, self.params['x'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**小批量梯度下降优化器** 复用3.1.4.3节定义的梯度下降优化器SimpleBatchGD。按照梯度下降的梯度更新公式$\\theta_t \\leftarrow \\theta_{t-1} - \\alpha \\mathbf g_t$进行梯度更新。\n",
    "\n",
    "**训练函数**  定义一个简易的训练函数，记录梯度下降过程中每轮的参数$\\bm x$和损失。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def train_f(model, optimizer, x_init, epoch):\n",
    "    \"\"\"\n",
    "    训练函数\n",
    "    输入：\n",
    "        - model：被优化函数\n",
    "        - optimizer：优化器\n",
    "        - x_init：x初始值\n",
    "        - epoch：训练回合数\n",
    "    \"\"\"\n",
    "    x = x_init\n",
    "    all_x = []\n",
    "    losses = []\n",
    "    for i in range(epoch):\n",
    "        all_x.append(x.numpy())\n",
    "        loss = model(x)\n",
    "        losses.append(loss)\n",
    "        model.backward()\n",
    "        optimizer.step()\n",
    "        x = model.params['x']\n",
    "    return paddle.to_tensor(all_x), losses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**可视化函数** 定义一个Visualization类，用于绘制$\\bm x$的更新轨迹。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Visualization(object):\n",
    "    def __init__(self):\n",
    "        \"\"\"\n",
    "        初始化可视化类\n",
    "        \"\"\"\n",
    "        # 只画出参数x1和x2在区间[-5, 5]的曲线部分\n",
    "        x1 = np.arange(-5, 5, 0.1)\n",
    "        x2 = np.arange(-5, 5, 0.1)\n",
    "        x1, x2 = np.meshgrid(x1, x2)\n",
    "        self.init_x = paddle.to_tensor([x1, x2])\n",
    "\n",
    "    def plot_2d(self, model, x, fig_name):\n",
    "        \"\"\"\n",
    "        可视化参数更新轨迹\n",
    "        \"\"\"\n",
    "        fig, ax = plt.subplots(figsize=(10, 6))\n",
    "        cp = ax.contourf(self.init_x[0], self.init_x[1], model(self.init_x.transpose([1, 0, 2])), colors=['#e4007f', '#f19ec2', '#e86096', '#eb7aaa', '#f6c8dc', '#f5f5f5', '#000000'])\n",
    "        c = ax.contour(self.init_x[0], self.init_x[1], model(self.init_x.transpose([1, 0, 2])), colors='black')\n",
    "        cbar = fig.colorbar(cp)\n",
    "        ax.plot(x[:, 0], x[:, 1], '-o', color='#000000')\n",
    "        ax.plot(0, 'r*', markersize=18, color='#fefefe')\n",
    "\n",
    "        ax.set_xlabel('$x1$')\n",
    "        ax.set_ylabel('$x2$')\n",
    "\n",
    "        ax.set_xlim((-2, 5))\n",
    "        ax.set_ylim((-2, 5))\n",
    "        plt.savefig(fig_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义train_and_plot_f函数，调用train_f和Visualization，训练模型并可视化参数更新轨迹。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def train_and_plot_f(model, optimizer, epoch, fig_name):\n",
    "    \"\"\"\n",
    "    训练模型并可视化参数更新轨迹\n",
    "    \"\"\"\n",
    "    # 设置x的初始值\n",
    "    x_init = paddle.to_tensor([3, 4], dtype='float32')\n",
    "    print('x1 initiate: {}, x2 initiate: {}'.format(x_init[0].numpy(), x_init[1].numpy()))\n",
    "    x, losses = train_f(model, optimizer, x_init, epoch)\n",
    "    losses = np.array(losses)\n",
    "\n",
    "    # 展示x1、x2的更新轨迹\n",
    "    vis = Visualization()\n",
    "    vis.plot_2d(model, x, fig_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**模型训练与可视化**  指定Sphere函数中$\\bm w$的值，实例化被优化函数，通过小批量梯度下降法更新参数，并可视化$\\bm x$的更新轨迹。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. \n",
      "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
      "  if data.dtype == np.object:\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x1 initiate: [3.], x2 initiate: [4.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAF5CAYAAACFu8BrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3WmUZOd93/fvc5e6tVd1V6+D2QAMVgIkQIIWLVK0djOyosjH8YmO5SW2ZdqxkshxFCdSnDDySRw5cRTLR3EU2PSxHcumHEmUJUumREogKe4iuAEgQGwDzNZ711519ycv7q3b1T3dM9ON6anunv+H5+Gz3FvVTzWArl89dymltUYIIYQQ4jgyJj0BIYQQQoiDkiAjhBBCiGNLgowQQgghji0JMkIIIYQ4tiTICCGEEOLYkiAjhBBCiGPLmvQEAJRSbwBdIAJCrfVTk52REEIIIQ7TXu/9Sqn/AvjxdPy3tNZ/60bPcySCTOq7tNbrk56EEEIIIe6Ybe/9SqnvAv4j4B1aa08pNXezJ5BDS0IIIYQ4Kv4z4Ge11h6A1nr1Zg84KkFGA7+rlHpWKfXBSU9GCCGEEIdut/f+B4HvUEp9USn1KaXUu2/2JEfl0NL7tNZX0yWkjyulXtJaf3p8h/RFfhCgVCq96+GHH57EPIUQQog77tlnn13XWs/eqZ+nlLod31/0AuCO9Z/WWj891r/uvZ8kl0wD7wHeDfwbpdR9+gbfp3QkgozW+mparyqlPgr8EeDTO/Z5Gnga4KmnntJf/vKX7/g8hRBCiElQSr056TkcgHuji3f2eO+/AvxaGly+pJSKgRlgba/nmfihJaVUSSlVGbWB7ween+yshBBCCHFYbvDe/+vAd6XjDwI54IYXAh2FFZl54KNKKUjm86+01h+b7JSOnziOiaKIKIoIw3DX9m5l9Ljd6lspWuvr+qOx8XqvAlzXHq/3I/13aFu9s71bMQxjWz1qj8rO/o2KaZp71nsVy7J2bRvGxD9nCCHEYdn1vV8plQP+qVLqecAH/sKNDisBqIO8YUzabTp2J4QQQhwpe70nK6WevZP3WLtN77N3ZM5HYUVGAO9///uxLAvLsrBtG9M0s3o0Pirjn9x31nt90r/RysD4ysFuqws7VydG43utaOy2wnGjAtevnhzUXqs8Nys7V5d2jo/aURRdtwq11yrW+ArXXithN1o5G43tVo+XKIoIguC68U9/+tM3+lUJIcSJcGKDzPvf/34cx8lKLpfb1h71R+1R37btXfuj2rKs7HG2bW/bPmqbpjnply/EgURRhO/7BEGQ1aPi+z6e5xGGIb7vZ2W0bbzved62sfG+53lZf9T2PE+ClxDiQI5lkHnyySf5zGc+M+lpCHHimKZJoVCgUChMeirXieM4Cz2u62b1zvaNynA4vK49HA75xCc+MemXJ4Q4oGMZZIQQdx/DMCYSsqIoYjgcMhgMsuAzGAy21f1+H9d1GQwG9Pv9bNvO8vGPf/yOzl2Iu4EEGSGEuAHTNCmXy5TL5UP7GWEYZiFoVI/Kzn6/36fX623rdzodPvnJTx7a/IQ4yiTICCHEhFmWRbVapVqt3tbn9X2fXq9Hr9ej2+3S7/fpdrtZf7zu9Xp0Op1s7JlnnrmtcxHisEiQEUKIEyqXyzE9Pc309PRbfq44jrMg1Ol0ttXtdptut7ut3el0aLfb/N7v/d5teCVC7E2CjBBCiJsyDINKpUKlUuHUqVMHfp4oiuh0OlnQGbVbrVY2NhoftSUMiRuRICOEEOKOMU2TqakppqamDvwcrutmIafVam2rm81m1h8VOUx2skmQEUIIcazk83ny+Tzz8/P7fmwcx9khsM3NzSzsNJvNrIz6m5ubfOpTnzqEVyBuJwkyQggh7hqGYVCr1ajVapw9e3Zfj/V9Pws7m5ubNJtNNjY2tvU3Nzf56Ec/euD5xXEs37O2TxJkhBBCiFuQy+WYn5/f90rQYDBgc3OTjY0NNjc3s7KxsZGVzc1NPv7xj0uIOQAJMkIIIcQhKhaLFItFTp8+PempnEgS/YQQQghxbEmQEUIIIcSxJUFGCCGEEMeWBBkhhBBCHFsSZIQQQghxbMlVS8eQ1po4jomiiDAMt5UoiraNj9qj8TiOt/XHx/eqd7bjOM7msLNorbNt4/V4AbLx0esZb4/X+6GUuq4eb48uaxyNj4phGNvqUXtn2TlumuZ17d3q8WJZ1rZxy7KyetQe748XuSxTCCGud9cHmTiO8TwvK77v4/t+1g+CIOsHQbBtn9G23fpBEGT9UR2G4baxURmNj7fH6/EyGhN3J8uysG37upAzGhuvd7Zt2yaXy2Xjo/aoHrXHx3frO46Dbds4jrNtH8dxtm0bFQlgQojDdCyDzNraGj/3cz+H53m4rstwOMzarutm7VEY2dkeBRPXdQ8lFIz+mO98g9jtjcUJoGQ6WKUytmUlxbSxLBPLtLAtG8s0sSwT27Qx03HLNNNiZbVhGtv6pmEkn/INA8MwMdPtpmFgGukqQdY2UMrIxozR+GgFQhnpPgrTMFGGwlBb25RK7pipGFvh2LHyoRitdmzvw/WrJwe11yqPZrRSpLf1sxWksZWkWI9WkSDW8dZYrIniCK01URRn26I4Joqjrfb4SlYcZWNRHBGmdRTFxOm2MAyz8TCKCKOQOIoJozDrJ3XSjsKIIAoIw2QsCINkn3Q8CEOCMCQMA8Igwlchw+HwurC8M3CPyu1mWRb5fH5buBmV8fFRe3T7+VG7UChsGx+V0XihUMj6O9u2bd/21yOEOFrUQZbwJ00plU3atu09/wDm83nyAeTsHHnHwbFzODkHJ5dL2tlY0s/ZOXK2vdUfG3fs9FOpbadjaUCxknbOtrEtC8u03tIbsRCTpLUmjJIg5AdBUsJ0VTEI8AIfPw0/XuDj+T5+mK5ipn0vSENRGGR9z/O2tvtJ2/U8/MDHtbnuQ8h4OwiCA78e0zQpFotZuBm1x+vR+Hh7fOxGpVQqUSgUZNVJ3DbFYnHXcaXUs1rrp+7UPMbfZ9+COzLnY7ki8/gDj/DMP/43OHYO0zQnPR0hTgylFLZlY1s2xXxh0tMBIIqiLPgMPRfXc3F9f6w9Gk/qoZuMDz2PgTfM2kN3mGxP292gy+rqKsPhkMFgkNUHWZUahZpSqUSxWKRcLmf1aLxUKl3Xr1Qq2+pyuZwVx3EO4bcpxMlzLIOMZZpH5o+sEOJwmaZJ0Szcsf/moyhi4A4ZuC5Db5i1B8MBfXeY1f3hgMFwuG1sMBzQGw7Sesja2hqDwYB+v5+VW2Xb9rZgMyqVSuW6etSuVqvX1aOQJKtG4qQ6lkFGCCEOi2maVEplKqXybX/uOI4ZuEN6wwH9QT+tB/SGffrDAd1Bn16/T2/Ypzfo0xsM6A36yfigR7ffZ3l5mV6vR6/Xo9vt3tJ5fkqpLPBUq9Us4IzXO9vVapVarbatnc/n5dC5OHIkyAghxB1iGAblYolysQSN2bf8fFprPN+n0+/RG/TSur9r3e330rpLp9ej3W5z+fJlOp0OnU7nllaLbNu+LtyM+rVajXq9no3t1q5UKrIyJG47CTJCCHFMKaXIOw55x2FuuvGWnisMQ7qDPp1el06/R6ef1r0u7V436aftUen4Hq+88grtdptOp0Ov17vpfEfhZhRwRqVWqzE1NbVtbHp6mnq9no1blrxlievJvxVCCCGwLIupao2pau3AzxGGIZ1+j3avQ6vbGQs9HVqdNq1el3a3Q6vbTraHW0Go1WoxGAxu+PzVapWpqaldy/T09LZ6VBqNBrlc7sCvSRx9EmSEEELcFpZlMV2rM12rH+jxfuDT7IyCTpvNTjsJQN0Om50WrW6HZqfFZqdNczDg2rVrbGxs0Gq1bniuULlcZnp6Ogs5jUYj649Ko9HIxhuNBpVKRc4HOiYkyAghhDgScnaO+cYM842ZfT1Oa0130KPZbrPZadHstGl2Wmy0W2y2W2x2mmndZrPf5/Lly2xubtJsNvf8OhTbtrcFnL3KzMxMVheLRQk/EyBBRgghxLGmlKJaqlAtVTh36vQtPy6KIlq9DhutJOhstNPA026y0W6yPhoP+7z00ktsbGywsbFBHMe7Pl8+n8+CzSjc3KhMTU3Jyc+3gQQZIYQQdyXTNGnUpmjUpm75MXEc0+512Wg12WhvstbcZKPdZKPVZL21yXpab3S7XLx4kfX1dbrd7t4/v9FgdnY2Kx/5yEck3OyTBBkhhBDiFhmGkZ0UfYHzt/QY1/NYb2+y3twcCzybrG5upPU668MeL730koSYA5AgI4QQQhyivONwem6R03OLk57KiSTRTwghhBDHlgQZIYQQQhxbEmSEEEIIcWxJkBFCCCHEsXVkgoxSylRKfVUp9e8mPRchhDhKPvKx3+ChH34/pT/6AA/98Pv5yMd+Y9JTEuK22Pner5T6HqXUV5RSX1NKfUYpdeFmz3FkggzwE8CLk56EEEIcJR/52G/w4z/701xevobWmsvL1/jxn/1pCTPipNj53v9/Az+qtX4C+FfA377ZExyJy6+VUqeBPwH8L8DfnPB0jhWtNVEUEUYRYRReV0dhWsdxNj7aP46jrB/FMdFYP9YxURSj0zrpR8RaE8URcRwnRWvi9LE61miSvtaaWOukjuOx8WTOo/7oNWh0tm00tl+jW4MrpVAKFCobMwwj6yu11TcMA6UUhkq2ZfsZCtMwMQwDIx03DCMZUwrTNDGUgWkaqLQ2lIFlWphmsp9pGJimiWWaGEZSj/qWaWEa6f5W0h+Nj2rbsrL5ibvXh37x7zN03W1jQ9flQ7/49/mRD/zQhGYlxFu3x3u/BqppuwZcu9nzHIkgA/wD4G8Blb12UEp9EPggwJmFU4c+oSiK8AIfz/eT2vOyvut7+IGPHwTZmD9eBz5BEOCHQToWEIQBfpCWMN2e7hOEIWG6PQhDgjDED33CtJ2U0X4hYZSMhVFEEAaH/ru4k5RS2wLJrRoPQAcJQUeZbdlYpoltWUnQsZKQY1t2WidjOSuX9XO2jZVuz1k2OdvGtm1yVo6cnfbTcSeXS/YZ1XYuGUtrx06Kbdvkc87WmOMkdS5HPufIjbwOyZWVpX2NC3GEzCilvjzWf1pr/fRYf7f3/h8DflspNQQ6wHtu9kMmHmSUUj8IrGqtn1VKfede+6Uv/mmAR+57QH/p+a8y9FyGnofrugy8IUPPw/O8dNzFTduu7+H5Xjbm+f628aT203YSSG5nQDBNE8dO30Cs9A1l9EZi2cmbUNq32h4F08I2i1i2ie1Y2KaJZZjYRvJp3TaSvmWaWMrENk3M0VhWjGzMVEbWT9rJSkM2royxfrKSoJTCVOk2ZWRjxmjcSMaVUhgoDJWuXIxWNpQaWwHZ0Yf0scmqSfrvwW37fY9k4SZd7Yl1jGZsRSgNPTv7kY6z/WPSVSUdJ6tWOt7aL05XqtIyGgvjKNkv3T8cPW5sWxRH2XgYR1k/jKOsRHFEEEWEOl1xiyOCdNUsiMOkn7aDKCRwI8LII5oyaPe6WfBNwnIalNPwPArhURTdtt+3bdlpqMnhpIEnCT4O+ZxDwcknY86ondT5nJONFZw8juNQTNv5fJ6C41BwChScPMV8nryTp5gvUMznydm5E79idXp+kcvL138oPT0vN1cTh+fJJ5/kM5/5zFt6jlKptK61fmq3bTd47/+vgB/QWn9RKfXfAD9HEm72NPEgA7wX+CGl1A8AeaCqlPqXWus/u9cDXrr4Kt/5Y3/6pk88+kOaH/1RTP/AFpw8dsenYdk4dpF8IYdj2TimTd7aaucsC8fM4VgWOdPGsWxyppVu2952TAt71DeTbTnLwjaSQwjizstWdlCgwDxSp4QdDVEcE8QhfhjiRyF+FOBFAUEU4kVJCPKiINk21vbCINk3DPEiHz8M8aIALwxwQ397Owjwhh6Dqmaj3cw+MCQfLJIPHX5wsA8OhmFkAafgJOGmWCiO9bfGSvkChXyBUqFAKV+kkM9TLpQoFpKxYr6YbSsVi5QLJZzc5IPSz/y1n+Sv/92fwvW9bKyQz/Mzf+0nJzgrId6y3d77fwt4WGv9xXSfXwY+drMnmniQ0Vr/FPBTAGkq+8kbhRiAc/V5/tfv/4sUrBwF2yFv58hbOQpWUid9G0PJG5cQN2IaBqaR/HczSbGOccMAN/BxQ59hmNRu4DMMvKw/TLcPAo9h4GVjg8BlmLaHvseg32ez7HFl5RoD12XoDum7QwbucF+HHg3DoFwoUiwUKReKlApFysVSWhcpFUrJeLFIpViiXChRLpYoF5P9KsUypUKRSqmU9ks4OWdfv5sf+cAP8Y1Xvsk/+KV/gkJxemGRn/lrPynnx4hjbbf3fuCHgWWl1INa65eB7+MWLgKaeJA5iHq+xPddeOekpyGEuE0MZVC0HYr2/t7k90trjRsGDAKXQeAx8L2s3Q88Bn46Hnj0fZdB4NL3XfppPQg8XNNko93k0tIVesMB/eGAbr9PGIW3NAfbsqkUS1RKZaqlchJw0vZ4XStXkn6xnJ0Y/1u/8C946Pz91MoVtNYTXy0S4nbSWodKqb8C/KpSKgaawF+62eOOVJDRWn8S+OSEpyGEOKGUUhTsHAU7R+M2Pq/WGj8K6flD+r5Lz3fp+cPtfW+YjiXbut4gqdtD1nyPi1cv0en36PS6DD1315/zA//5n8valmlthZ1ymXq5SrVcoVauUEvb9XIlravUKltjtUqVWrmCbdm38bcgxMGMv/drrT8KfHQ/jz9SQUYIIY4jpVRybp1l0yhWb/6AmwjjiK6XhJ2uP+RnP/XLfO7SN/nfPvBjdL0BnVFxkzDUdvv045g3r12h3evS7nXo9Hs3PYxWKhSplStMVWvUytVt7Xq1ylSlRr1So1apZO16tcp0tU7BycuKkDgSJMgIIcQRYxkmU4UyU4UykFx5d6Y2y5962/tu+TliHdPzXDregLbXp+326bhb7a43oOVujfcsg2trK3zz9Vdoddu0e90bPn/OtqlXakxVakxVx0s9a09n7TrTaV0rVzBN8y39foQYJ0FGCCGOuOVuk4XK9L4eYyiDar5INV/kNDP7/plRHG8LOy23d127NUzrTo9r3govvPYyrW6bTr+35/MqpbLwM12rM1Wt06jVma5NMV2rM12t0xi1a3Vm6lM0atMU8vl9vwZxd5AgI4QQR9xSd5PH5s/f0Z9pGgb1Qpl6uiq0H0EU0nYHtNwezWGPptujNUzaLbfH5rBLc9ilOeixFm3wrTdeY7PdpDvo7/mcxXyBRn2KRm2KmfoUM/VpGvVpGrUpGvUpZuvTzExN00i3TVfrsvJzl5AgI4QQR1gYR6z12yzuc0VmkmzTYqZUZaa0v/OF/ChMgs+wy+awy+agm7Z7bAw6ydiwS6vX5fWrl1hvbu65+qOUYrpaZ6Y+zez0NDP1aWbqDWamppmtTzM71UjHG8xONWjUJPgcVxJkhBDiCFvrt9FoFspTk57KocuZFvPlOvPl+i0/xo9CNoddNgYdNvodNkbttKwPOrSIeeniq6y3vsRGu7XrSdBKKRq1ehJwphrMTc9k7dnpBnPTDebS8bnpGUqF4u186eItkCAjhBBH2FJ3E4CFyskPMgeRMy0WylO3HPSiOM6Cz3q/zXoadtb7bTYGHdb6bZpRxNdf/iZrzQ1a3c6uz1PMF9JQsxVuRv2F6dmsP9+YoVwsyRVeh0iCjBBCHGHLoyBTPj6Hlo4y0zCYLdWYLdVg9sxN9/fCIAs4a4M2a/026/2kXuu3WOu1eWN4mS8991XWWpu7rvaMQs98Y4b5NOTMN2ZYmJljoTGb1XPTDQqH8aJPOAkyQghxhC33mgD7vmpJ3B6OZXOq2uBU9ea3UIzimI1Bh9V+i7V+O6vX+i1Wey02nJjXrrzBZ7/2h2y0m9c93jItXM+Vc3X2SYKMEEIcYcvdJqYymLkNN9oTh8s0DObKdeZu4RwfPwpZ67dY6bVY6TVZ7bXoeAMJMQcgQUYIIY6wpd4mc+U6piFfgnuS5EyLe6oz3FPd/z1+xHbyX4YQQhxhy93msbr0Wog7TYKMEEIcYSu9zbvi0mshDkqCjBBCHGFL3U3mJcgIsScJMkIIcUQNA5+W25crloS4AQkyQghxRK2kl17LOTJC7E2CjBBCHFHZPWTk0JIQe5IgI4QQR1T29QQSZITYk9xH5gTRWhNrTRCHhHFMGKX1qB9HRHGUtHWU9uOsjuKISMeEY+0ojol0TDyqtU7rZCxOf+ZoTGvQbI3H6WNGcwOd9XU6Z41OH5f8L3ktY6+L62/5PaLY+v6S0VeZqNH/VNpWyV5KKQylgKQe9Q1lpLVCYaAUGMrAHI0bRrbfaMxUBoaR9M2sNrHG2qZhYI3Vlkq3G2bSNwwsw0pqM6ltw8rmJsRKb/Q9S3JoSYi9SJDZp1jHeGGIF/q4UYAfBnhhgBsFeKGfbIt8/DDEDX38KMRL9xu1g2wsxI/CrB9EaT8O8cKQMN7aHsRRtj1M22Ec7ejHN3zTP4pGQWRn6EjG1HX7jRt/raPvN9kZjnbudxwoVBJqTCspafAZ74/XuXQ8Z1rYxlbfMe1sPGdt9R3TTsdsHNPGSetknxyOZeFYOfLptpxlp+1km6FkIfdOWe42yVs29Xxp0lMR4sg6EUHGj0KGgccgLcPAYxj4SR16DNK2G/q44WjcZxj4uEE6lm4b9d0wwA19vHSbl/b9KLwtc7YMk5xp4Vj2rm8+tmmSM21yEZQMh5xlJW9mY8UyTHKjN7nxWo2vAIw++W+Nm8rIVg6yvjIw0nFLmRhGsupgpuPmaBVix8rEaHVCKYXBztWNHXW6z3houVPGw03M1gqRZns9Wl3S6T6jVahtK1FpO0pXnCI9WrXS6UpXsl+YjofjK11pPxzV8aifrJAFY/0g3iphfH3fj0OCKCJUMYOgTxBF14ViPwrSOgm8t0MWfiybgpVLQo+VI28n4SdvJ/1COl6wk20FK0fBdtKxpC7aOQqWQzHnULCdpG0n24u2g22eiD9RB7bca7JQnpYVOiFu4Fj+lXhx7RKP/cO/moWXg/yBtgxz649t+oc3+2McK+p2iXxhCse0KJg5HNMin9XpJ9k0eOTTdi4bs8gZY+2xfs6wyJmmfKq9w1QaqFBwt36TSaxj/CgJQF4U4sejVcKtftaOkrYbBXjRaDxdeYxC3MjHi0KGae2mK5QuAS23n31ocIOtDwkH/e+0aDtZsCnaDsVcntJ4385Tyo2VPfrlXJ5yrkDZKZA7JgFJ7iEjxM0dj/+adyiaOb538VGKaRAZ1dva5tZY3rSzbXnTJm/lsI279e1M3K0MZZC3DPLYE/n5QRxlgWcwWgWNgmR1NPQZRkk92rZVe1m7n7a78ZCVXjNZhfU9+oHLIPBueS62YVJ2CkmwSQNOadR2ClRyBSpOgYpTpJzLU3GKVNIQNGqPth9mKFruNXn7/L2H9vxCnATHMsicrczwf7z3Ryc9DSHEPtiGiZ0rUKFwKM8f65hhGDAIPfqBRz/06AUu/cCjNxoL3Gy8F4y2J+2O1+daZ52e79Lzh3S94S2dX5W3bCq5ImWnQNUpZqWS32rX8kUqo7ZTopovUnVK1NJ9djuEprVmubvJ919452H8uoQ4MY5lkBFCiJ0MZVCyHUq2w+xtyEpaawahRy/w6AYu3WBIP/Do+MOs3/XHazfZFgWsN5foeAM63oCuN7zpzyraThp4SlTzJWpOkZLtMAg8Xlh5k3/0xd+k6pSo55NSK5SzdjlXkHNoxF1NgowQQuxCKZWcX2Pnmad24OeJ4phe4NIJhnT9IW1/SDdI6k5a2v4grYd0/AFr/Tbf7LcB+Mybz/MHbz6/5/ObykhCTaFMPV9O61LSzpeYKlSYKiTjU/ly0s6XqRdKWHKIXZwAEmSEEOIQmYZBzSlSc4r7etwfXHuJP/27/5Bf+eM/wdsbZ2n5A9reIKl3tJte0m/7AzYGHV7bvEZr2KPtDm54eKzqFJlOg850scp0ocx0oTLWrtIoVraN3e1XkomjR/6NFEKII2h5mKzILJamkpOLcwXOlBv7eo5Yx7T9IS2vT9Mb0PL6tPwBm16Pljeg6fWT4vZZ77d5Zf0KG8Mufd/d8zmrTpFGMQk4jWKV6UKFmWI1HUvKeL+Uy7+l34MQNyNBRgghjqDlfguA+UL1wM9hKIMpp8SUU2I/1z55UUDT67Pp9tj0+mx6fTbcLk2vz4bbY9PtseH2uNrZ4LnlN1gftPe8x1bRdq4LN7OlGjPFKjOlWtpO6kaxemwujRdHh/wbI4QQR9DysE0lPUfnTnNMm4VinYVi/Zb211rTDz023B4bbpf1NOisu1023G7WXu23eHHtEuv9Dl4U7Ppc9XwpCzazpRozpRpzpRqzpXo2NmoXbed2vmxxTEmQEUKII2h50L7lIDFpSinKdp6ynedcZeam+2ut6QUu6243KcMk/Ky7XdaGHdbcLuvRkBfXLrP2xnO03P6uz1POFZgt1Zgr1Zkr15lLA85cuc58qc58eYr5cp2ZUk1ObD7BJMgIIcQRtDJosVA8+NVSR5lSKjvv597q3E3396MwDTld1twOq8MO68MOa8Nu0o5dXlq7zKffeI72LqFHoZgpVZkvT7FQnsrqhcoUC+VpFirJ2KwEnmNJgowQQhxBS4M2f3ThwqSncSTkTItTpSlOlW7+dQ1uGCRhZ9Bhze2wMmizMhzVbVb7Lb6xfJH1QZtYb7+iS6GYLdVYrEynIWeKxco082nYWaxMs1ieZqpQlnv3HCESZIQQ4oiJdczqsM3iMTm0dJTkLZsz5cZNr/AK44h1t8vyoJ2FnOVBm+VBi+VBmyvtdb585WU2ht1df8ZCeToJNllpsFiZ5lR1mlOVBnPluqzu3CESZIQQ4ojZ9PoEccT8CT20dBRYhnlLJzR7UcDqsMPyoMXSoM1yv8XSoJUEnrDPV5ZeZfnlTdxw+8nLhlLMleqcqiYB557qDKcqSchZrDa4p9pgoTwl9+W5DeQ3KIQQR8zo0uuFgqzITJpj3nyFR2tN0+uzPGhxbdBiKQ071/pNlgctXt24xqcuPkfP3/51FQrFXLnOPdUGpypJuPmFn/mTWJa8Ne+H/LaEEOKIGd1Qx2tSAAAgAElEQVQM76Se7HvSKKWYzpeZzpd5dPr0nvt1/SHX0oCz1G9ytd/k2qDJtX6Tb61f4Q+vvswvSojZt4n/xpRSeeDTgEMyn1/RWn9osrMSQojJWRmMgoysyJwklVyBh3IFHqovTnoqR4ZSygS+DFzVWv+gUupe4CNAA3gW+HNaa/9Gz2Ec/jRvygO+W2v9DuAJ4ANKqfdMeE5CCDExy2mQmXsLd/UV4pj4CeDFsf7fA/5PrfUFoAn85Zs9wcSDjE700q6dlr2/5UwIIU645UGLRr4st+sXJ5pS6jTwJ4B/kvYV8N3Ar6S7/HPgh2/2PBMPMpAsLSmlvgasAh/XWn9x0nMSQohJWR605ERfcTf4B8DfAuK03wBaWuvRF3ddAe652ZMcibivtY6AJ5RSdeCjSqnHtNbPj++jlPog8EGA06XpCcxSCCHujJVhm4WSnOgrJmgYoJ5beqvPMqOU+vJY/2mt9dMASqkfBFa11s8qpb7zrfyQIxFkRrTWLaXUM8AHgOd3bHsaeBrgHTPn5NCTEOLEWh60eXz67KSnIcRbta61fmqPbe8Ffkgp9QNAHqgCPw/UlVJWuipzGrh6sx8y8UNLSqnZdCUGpVQB+D7gpcnOSgghJiOII9aGXbkZnjjRtNY/pbU+rbU+D/wI8Pta6x8FngH+43S3vwD825s918SDDLAIPKOU+gbwhyTnyPy7Cc9JCCEmYm3YQaPl6wnE3eq/Bf6mUupVknNmPnyzB0z80JLW+hvAk5OehxBCHAWjS69lRUbcLbTWnwQ+mbZfB/7Ifh5/FFZkhBBCpJYH6dcTSJAR4pZIkBFCiCNktCKzUJAgI8StmPihJXFzURzjxSF+FBLEIV5a+3FEEIX429oRYRzhxyFhHBGMlXC81kk7iuOk1jFhHBGO6jgm1nHWH7UjHRPFaa1jYq2zsRhNrDVa62w8RqPT/TQQa0081tc6Tmo0Or0Wbbw96u9FobbaKukrtbVNAUoZGCppG9vaCqUMTKUwSMeVwlAKA4VpGJjp/qZK2qMxSxmYhpm1DWVgGQaWYWKppDaVgW2YmEZS28rEMkxsY6veKtb2vmmRM0xyhpW1bcPCMS1swyJnWjiGhWnIZ5GTZmXYwlQGM4XKpKcixLEgQWYfYh3jRiHD0N8qkb+t70YBwyjI2m4Y4EY+XhTihj5uFOJFAW4UZLUfJeHEj4Nsux+FWXiJdHzzyR2QPfbGO3rztQwDU5lpbYyNpW/YY2/o9lQFZ9Q3ku2GoTCUkYaC7WPjYWG0HZIvXVNKZcFEqWRsZDywjIwHHK2T8DMa02mgGv1zG4WrUR3rmDgeharR9mQsCWpRtj1odgnigChOwlychrhwLNCFWRhM+klITIJhEEeH9s/PVEYWahzTJmda5E2LnGHjmFbat3FMe6y2yFs5HNMib+bIWzaFrE72K1i5rWLmtvXzppX9cxO33/KgzXyhKr9jIW7RiQwyURzTDz16gUs/8OiHXlYPdvT7gccgTLeFPoPQYxD49EOPYegzyEKKxzAKDjQf2zDJZ28yOQrpG4pjWhQbdUp2DseycewcjpVLatsmZ9lpP2mPimNt7+csC9tM6pxlY5lWOpaM25aZti0sczRuYhrmtrAgDofWmiiOCKJk1SyMQoJRCSOCKEhW08Jkmx8G+GFIECV10k+KN6oDnyAK8YIAL/TTsWTcC/20DnB9j85mGy/q4kZJkHYjHz8KcaPgwCFrFHCKabgpWjmKlkPRTuqS5WRjJTvpj+rijn7ZzmdtWWFKzpGZlyuWhLhlxzLIXOlt8Fc/+WF6YRJUeoFLLxgFF3dfgSNnWBStHCXbSf8oJ/X8qUWKToFizqHoFCjkHEpZnaeQy1OwHQqOQyGXp5jLk8/lktp2KOSSkrdzWPJ9KXc1pRRWGiILOJOezjZhFDL0fdzAY+h7uIHHwHdxfZ+B7zL0XYZeOhZ49D2Xoe/R94YMfY+BN2SQ1t3VDdaGXYbhevoBIfkg4MfhzSeSKpg2JTtP2c5Ttp20TkolDTwVu0DFzlPObY1X7DyVXDJezRUoWc6xDenLgzb3VecmPQ0hjo1j+Q7b8Yc8t3k5+2N35uxZKvkSpXyBcr5IJV+knC9ScpJ+2SlQLhQp5vKU8wVKToFSvkApV8C2juWvQIjbwjItKgWLSqF4aD8jCEP6/pC+O6TvDell9YC+O6Q3antDuu6A3lhpL6+xOmzzemeVru/SC12GoX/Tn2kolYQdu0A1l6dqF6jmklLJFbJ+LVdM6wLVXHFbPakvbFwZtPn2hQcn8rOFOI6O5bv4Y2cf4FMf+seTnoYQ4hbYlkXdqlAv3p6TV8MopOcN6Q4HdId9um6fzqBPJ62z/rBPZ9ijOxzQHvZYXl3nW60luoFLxx/e9NyzgpWjnitSS4NNzUna9VyRulOi5hSy7VNOibpTou4k2y3DPNBrG4Y+LX8gl14LsQ/HMsgIIe5elmlRL761YKS1pucO6Qx7tAdjZdij1e+m/S6tQY/WoMvmtWWu9pq84F+l7Q/oBe4Nn79qF6g7ScCZdspM50tMOSWmxtoNp8x0vpxtd0ybldHN8OTSayFumQQZIcRdRylFpVCkUihyz/T+z0cJo5D2oEez36XV79Lsd2gNkrrZ77LZa7PZ67DZ67C+vMzr3VU23R7dGwSgsp2naOUA+GcvfZrPLr9MI1+mka8ktVNmJm3PFCrH+jwgIW4nCTJCCLFPlmnRqNRpVPZ3dZEfBjT7HTa6bTb7HTa6LTZ67aTfa/PVN77F6ivfYBB6fG75FTbd7p4XLziGxUyhkoWdmbTMFsbb1aRdqGAf8HCXEEedBBkhhLhDcpbNfK3BfK2x6/Zf+J1f5vOvfIPf/Z8/zFQpOXTW94asd1usd1tspPVap5WONVnvtli+eo2XW8usDzt4e1wlNuWUtgWduUKVuUKVmbSeK1SZzVcl9IhjR4KMEEIcEUutdfJ2jnqxnI2VnORKy3Mzizd9/Ojcn7Vuk7XOJqvtZtpustreZLXTZOXyFZ7buMya293zXJ9pp8x8MQ03hSrzhRpzaT1frLFQrDFXqFGyj9btBMTdSYKMEEIcEcutDRanZg987sv4uT/3zd1z0/0HnstqZ5O1TpOV9iarnU1WWhtJ4GlvsHT5Cq+2V1kbdna9H1DVLmTBZr5YY7FYZ6FY31bPFioHvopLiFshQUYIIY6I5dY6i/XdDzsdhqKT5/zsKc7Pnrrhflprmv0uK+0NVtqbLDXXWGqts9za4FprjWuXLvP55VdZGbQId1zWbijFfKHGQrHOqdJWyDlVmmIxrReKNRzTPsyXKk4wCTJCCHFEXGuu8cT5hyY9jesopZguV5kuV3nknnv33C+OY9a7La611lhqrnOtuZ6FnqvNVV65epVPX3tp16u3ZvKVNOhMcaqUlHvS+lRxisVSXc7dEbuSICOEEEeA1jo5tFSfmfRUDswwDOZq08zVpnni3N6BrDscsNRa4+rmGteaa1zdXOVqc5VrzTUuXb7EF1Zeoe0Ptz1GoZgvVrmnNJ2FnNNZe5rT5WmmnZJckn4XkiAjhBBHQGfYZ+C7xzrI3KrkPJ5zPLh4bs99eu6Aq5urXNlc3VZfeu11vrl5hY9ffg53x6XpBSvH6bFgc6bc4Ey5wdm0nitUJeicQBJkhBDiCFhqrgPcFUHmVpTzRR46dZ6HTp3fdbvWms1em8ubK1zZWOXyxjJXNla4tLHCpTff4LlLl9lwe9sekzdtTpfSgFNJAs65ygxnyzOcrTSo54oSdI4hCTJCCHEELLWSILMwJUHmViilspsS7nUYa+C5XFpf5tLGEm+uLfHmxjJvri3xxhuv87WNN2l6/W37V+w85yozWbjZajc4XZ6WE5KPKAkyQghxBCy3ZEXmdis6eR6+5zwP33N+1+3tQY8319OQs76UtV9+8yKfuPz8tpsLKhSnSlOcT8NNUs9yvjLL+coMNefwvkFe3JgEGSGEOAKyFZk7ePn13a5WLPP2sw/w9rMPXLctjmNW2pu8sXaNN9aucTGtX3v1FX738nOsu91t+085pSzUnK8mAefeyiz3VmeZyVfkkNUhkiAjhBBHwFJrnVqhTMkpTHoqguQKrMWpGRanZvijD779uu09d8Aba0tcXL3KxbWrXFy9xqvf+hZfXrvIv33jWWKts33Ldp77qnPcX51L6tp81q/k5J/3WyVBRgghjoCl1rqsxhwj5XyRx87cz2Nn7r9umx8GXFpf5vXVK7y+cpXXVq/wyosv8ezaRX794rNotkLOXKHK/dV57q/NcX91np/2/zK5XO5OvpRjT4KMEEIcAaOvJxDHX86yubBwhgsLZ67b5gV+snqzcplXli/x6vJlXn75W/z7S1/HDQM+ZMsJxfslQUYIIY6A5dY673voiUlPQxwyx87teQJydziQc2kOwJj0BIQQ4m4Xx3F6aEmuWLqbVQpy5dNBSJARQogJ2+i1CaNILr0W4gAkyAghxISNLr1elJvhCbFvEmSEEGLC5OsJhDg4CTJCCDFhWzfDkyAjxH5JkBFCiAkbfT3BQk3uIyPEfkmQEUKICVtqrTNbncK25I4YQuyXBBkhhJiwpaZcei3EQUmQEUKICVtqrbMoX08gxIFIkBFCiAlbbm2wWJevJxDiICTICCHEBAVhyFq3KSsyQhyQBBkhhJiglc4GWms5R0aIA5p4kFFKnVFKPaOU+qZS6gWl1E9Mek5CCHGnLDc3ALkZnri7KKXySqkvKaW+nr73/0w6/ktKqW8ppZ5XSv1TpdRNvw584kEGCIH/Wmv9KPAe4MeVUo9OeE5CCHFHyNcTiLuUB3y31vodwBPAB5RS7wF+CXgYeBwoAD92syea+E0LtNZLwFLa7iqlXgTuAb450YmJ68RxTBCFBFFIGEdEUZS1gzAiikclJopjwjgiHuvHelRrojhCa02sY7QmrTWx1mit0WgAtE7rXeYz+rL70dfeKxSGoVAolFIYykAp0lphGiaGUpiGgaEMTCMphmFiGWbWNw0T0zCxrWTcNi1Mc6ttmxaGcRQ+A4iTYKm1BsiKjLi76OSPey/t2mnRWuvfHu2jlPoScPpmzzXxIDNOKXUeeBL44i7bPgh8EOBMY/6OzmvSojhi4HkMfZeB7+L6PsPAw/W9pB94DH0fL/BxAx8v8BgGo76HF/h4YZDUQYAX+vhhgBcE+GFaohA/8PGjkCAM8MMQPwoIw5AgDSyxjif9qzgyDGWkocbEtmxs0yJnWdiWTc60yNm5pLZscpaNY6e1lcOxbRw7h2Mldd52yNs58nYOJ207do5CLkchl6dgO+RzDgXboeA4FHN5CjmHQs7BNMxJ/yrEW7TU2sAyTWYq9UlPRYjbbUYp9eWx/tNa66dHHaWUCTwLXAD+L631F8e22cCfA256usmRCTJKqTLwq8Df0Fp3dm5PX/zTAE+ef3i3D+gTF8cxPW9Izx3Qcwd007rnDui7Q3rekL47pO8N6XtuUrtD+v6QgeembZeBN2SYhpSh5+GF/oHn5FjJG2TOtpPasnEsm1z6hmq+2qOCRQ4TmxI2Jg4mFgY2JvZYe2dtYmCNFTMralvbGBtTqHRMpb2dNWkP2Pb/ZKPA1orNjhGNJr6undQxmijtR8REaOK0TvpJOyQmJibcVqKsHRAR6pggjAjDGN+Lsu0eEUFaPHwChrgXyrQHPfxwLFDuCJZv5Z/veLgpOgXKToGik6fo5CnlCpTyBUpOgXJaF508ZScZL+eLlPNFKmldzhcpOwVZcbqDllvrLNQa8jsXR0rcGtD/ja+81adZ11o/tddGrXUEPKGUqgMfVUo9prV+Pt38j4BPa63/4GY/5EgEmTR5/SrwS1rrX5vEHLzApzPs0x70aA+6tAY9OsN+UrJ2j+5wQMft0x0O6A77dMfaPW94yz+vkHOyN5WSU6CYy+NcHDJPjiI1CtgUyVHA3lbyWGmdtHerHSycNJwYoUrOQrr1qYnb7dUbb47R+ER4hHiEuAS4e9TDtD1M28MwKYO+z5CAAQEDOrTuDbjWXKPvpSE5Dce3quwUqBRKVApFKvlS1q7mS1QLSb9aKFEtlpO6UKJeLFMrVqilY46de4u/uLvDUkvu6ivublrrllLqGeADwPNKqQ8Bs8BfvZXHTzzIqOQEhw8DL2qtf+6tPJcbeLT6XZr9Ls1+h2a/Q6vfS+surUFS2oNe1k+CSw83uPGnYqUU1Xz6xzz9Q165GHCKCmUaVHAo4VAml9VlHErkKKXtYtouYmP6BvhA9628YnESGKg0hN7G/xwvXj8UETMgoI/PAJ8eHr0d7f6o9jy6nkev5dPFY3jBZbW9kYX7rjvIzl/aS97OUSuWqRXL1IsV6qUK9TTojPpTpSpTpe3tqVL1rgpBS811LiycmfQ0hLijlFKzQJCGmALwfcDfU0r9GPDHge/R+tbOZ5h4kAHeS3Ic7Dml1NfSsZ8eP+FnpyubK/yFf/QhNvttmr0Om702zX6Xge/u+UOUUlt/QIsVKm8GPEiZCg1qFKiRp7qtOFTG6pLOYQyVrGyIY8vEoIJDBWf/D96xqhSj0/Dj0cGjg0sXlzYu3bTfDlza7SGdtkebAavnQl5eupR9gLhRECrm8kyXq0m4KVeZLtVoVGo0ykmZqdRpVOo0Kmm7XDu24Weptc53PPzkpKchxJ22CPzz9DwZA/g3Wut/p5QKgTeBz6cXcvya1vrv3OiJJh5ktNafgbGTH25Bs9/lhSuvMV2usnDZ4BHuYYoC9R2lRj6tC1S0g9FX0D+kFyLEXcRAZaH/1K0+6M2tZoymi0ebIa20tHFpMaTJgJY/pLXp0twc0Lng88KV19jotWn2O3sGoFqhzEy1zkylzmxliplqUs9Wt8pcdYrZ6jRTpUp2tdskDTyX9qDH4pR8PYG4u2itv0Fycc/O8X3nkokHmYN4WzzHby//xUlPQwhxQAaKGnlq5DnL1I13HlsNiohp47JBnw0GbNCnyYB1+mwMB2wOBzSnc7y+eoUvvfYC693WrlfbWabJXHU6KbUp5qrTzNcazNemmatNM19L+gv1BiWncJtf/ZbsHjLy9QRCHNixDDJCiLuTicE0RaYp8sBeO7241YzRtBiyRo91+lm9HvVZa/ZYa/ZZObfJc5deZbXTJIqj656u7BSYrzdYqDWYrzdYrM+wkPYX6jNpf4ZKobjv17OcBhk52VeIg5MgI4Q4sQxUFnwe2mun9JDXKPSs0GWNHqv0WKHLqtdjbaXHRg2+/ubLfOxrn9v1fLyyU2BxapbFqRlO1ZN6sT7DqalZTk3Pcs/ULLPVqW33/llqJV9PcEq++VqIA5MgI4QQbA89j7DLTTdfTiqNpofPCt2sLNNlxeuyvNxlreTx2Ze/xlJrnTDavsJjmSan6rPcMz3H6cYcq+1NAF68+joazenp+QOt7AhxN5MgI4QQ+6BQ2dVfF9jlkNBrSRWj2aDPMl2W6LBMh2tRh2sbHVYaii+9+gKXNpYB+E9/8X/KHl4vVTjTWOBcY4EzjXnOzixwprHA2ZkFzs4sMlWq3IFXKcTxIUFGCCEOgYFiljKzlHmcxe0b09Wdv86v8BWu8Av8Ka7Q4iptrvbbXOm3eTW8zDPf/DL9HTfarBXKnJ1Z4NzMYlafm13M6sM8OVmIo0iCjBBCTMgKPe6hzlOc4Sl23BTvWnIYq8mQK7S4RIurtLg0bHHlcotXo8v8/gt/eN35OnPVac7PJqHm/Mwpzs+e4t65U9w7d498FYI4kSTICCHEhKzQ5R03uBOPGjtv5+0790uDzgYDLtHkMi0u0eTNTpNr9yi++Orz/OoXf3/b5ed5O8f52STc3Dd3T1LmT3Nh4Qynp+fkS0jFsSRBRgghJkCjWabD9/PggZ9DoZihxAwl3snprQ3pJegBEVdo8yabScgJmrxxrcllY5k/eOmr2w5bOVaOe+dOcWH+DBcWzvDAwlkeXDzLhYWzTJerB56jEIdNgowQQkxABw+XkHkO7+RdG5N7meZeprdvuJIEqRV6vMEmr7PB6+EGF69t8LJ+k9/5xucJojDbvVGu8cDCWR5YTMLNgwvnePDUOc7NLMgqjpg4CTJCCDEBK+k3xi4wmdUOhWKBCgtUeA/ntjYsQUjMJZq8zgavsc5rvQ3eNCI+9vXP8f/+wW9lu+Ysmwvzp3lg8RwPnzrPg4vneGjxHA8sniFvH+A7vYQ4AAkyQggxAct0AA51ReagLAzuo8F9NPje0aGv9EqrJkNeZ51XWOfVcJ1Xr67zXPAKv/nsp7PzcQxlcG52kYdPneOhxfM8fOo8D99zLw8tnqPo5Cf0qsRJJUFGCCEmYLQicxSDzI1MUeBdnOFd41dZrYJLyOts8CprvKzXeHV1ndfsJT7x3Jeyw1RKKc42Fnjknnt59PR9PHbmfh47fT/3z5/GMuXtSBzMLf2bo5R6GLgH+KLWujc2/gGt9ccOa3JCCHFSbR1aOl5BZi95LB5lnkfH74p8NTnh+A02eTkNOK+sr/GKs8Qnnv9idufjvJ3j4VP3JsHmzP08fvYBHjtzP/XiyfjdiMN10yCjlPovgR8nOQ/+w0qpn9Ba/9t0898FJMgIIcQ+LdOlRp4C9qSncqhsTB5glgeY5U+MBq+CR8irrPMiK3wzWOblUsjHvv45/uVnfjt77NnGAo+fvcBjZy7w+JkLPH72AudmFlFKTeS1iKPpVlZk/grwLq11Tyl1HvgVpdR5rfXPA/JvkxBCHMAy3WN3WOl2crB4Gwu8jQXgHfDN5EqqVXp8kxVeYJkXN1Z4MXeJ3/7aZ9FaA1Arlnn72Qe2lQcXz8qhqbvYrfyTN0aHk7TWbyilvpMkzJxDgowQQhzIyl0eZHajUMxTYZ4K38WFZHAJBvi8xCovsMwLg2Ve9AZ8+Jlfxw18IDk09diZC7zj3INpeYBHTt2LY+cm+GrEnXIrQWZFKfWE1vprAOnKzA8C/xR4/FBnJ4QQJ9QK3d2/dFJcp0iOd3J666Z/F5NLxF9jnedZ5oVgiRdzHv/fFz7Oh5/5dSC5NPyxM/fz1H2P8q57H+Gp+x7l/vnTcljqBLqVIPPngXB8QGsdAn9eKfX/HMqshBDiBIvTQyiyInNwFgYPMcdDzPGneDu8lPxe36TJ8yzx9fAaz+cG/NJn/j1P/96vAck3i49Czbvve5R33feo3LX4BLhpkNFaXwFQSv088Df06EBlsu2zhzg3IYQ4kTboExJLkLnNDFR2J+P/kLfBtyAi5mXW+CpX+Wr/Cs+1Nvjff/NfZPe8uTB/hnff/yhP3f823n3fo7zt9H1yvs0xs59/Wl3gN5RSP6K17iul/jjwP2qt33tIcxNCiBPpuN5D5jgyMXiEeR5hnj/DO+EK9PH5Otf4Clf46soVPjH8Ev/6c78DQDGX5133PcK7738b33b/Y7z7/kdpVOoTfhXiRm45yGit/7ZS6s8An1RK+UAP+O8ObWZCCHFCnbR7yBw3JXJ8O+f5ds4DoDuay7R4lit8xb/C19wOP//v/zVRnNzn5v7503zbhcf4I/c/xvseeoILC2fkXJsj5JaDjFLqe0guxe4Di8Bf0lp/67AmJoQQJ9WSBJkjRaE4yxRnmeJP8ji8AUMCvs41vsxlvrJyhd/pf55/9dnktmlz1Wne+9A7eO9DT/C+h57g4VPnJdhM0H4OLf33wP+gtf6MUupx4JeVUn9Ta/37hzQ3IYQ4kVboooBZypOeithDAZv3cC77Qk3d07zOBl/gTb7QeZMvvfo8H/3DZwCYqdR530NP8r6Hn+D9Dz/Jg4vnJNjcQfs5tPTdY+3nlFL/AfCrwLcfxsSEEOKkWqHLDCVszElPRdwiheJ+ZrifGX6Ud6Gbmku0+AJv8Pnum3zh9Rf49S8nwWa2OsX7HnqC73j4nfyxR94pl30fsgOfmq21XkoPNwkhhNgHuRne8adQnGOKc0zxn/AkejMJNp/nDT7XeYMvjK3YnJqa5TsefpL3P/xO3v/IOzk7szDh2Z8sb+kaM6318HZNRAgh7hbLdOX8mBNmPNj8CE+im5qLbPI53uBzzYv83vNf4pc//7sAnJ89xXc++i6+97Fv4489+i6qhdKEZ3+8ycXyQghxh63Q5R2cmvQ0xCFSKO6jwX00+LO8C93VvMwan+Uin127yK9+8ff4Z5/6TSzT5D0XHud7H/82vvexb+Pb9fvkMNQ+/f/t3Xt4XPV95/H3V6O56TbWxZIt+X63scEuNteEm4ElNECdwCbZNmkhWTZN0oYkXUo2uw9pm+cpTfbpwlOSbR3K8rRNaRKCm2sDCSGlhDvmYsDcjLEty7Is666Z0YxmfvvHjIRsy7IkS3N0NJ/X85znXOfM99iyz0fn/M7vKMiIiBRQmgzt9OuKTJExbLgn4ps4l3Qyw3Mc4NeZt3ksfpivPvB3fG3HPbTf8iFisZjX5fqKgoyISAG10QeoM7xiFyTA+SzhfJbw5QO5242vZA4pxEyCgoyISAG1qg8ZGcU8KvUzMUklXhcgIlJM9HoCkamlICMiUkDvvZ5Ab10WmQoKMiIiBdRKD6WUUEOZ16WIzAoKMiIiBXSYPuqpoAQ9YisyFRRkREQK6DC9uq0kMoUUZERECkivJxCZWgoyIiIF1EoPDXrrtciUmRFBxszuNbM2M3vF61pERKZLnBQ9DKi/ECl6ZrbQzB41s9fM7FUz+/xx679kZs7M6k61rxkRZID7gKu8LkJEZDodzvfqqzYyIgwCX3LOrQPOAz5rZusgF3KAK4H949nRjAgyzrnHgA6v6xARmU6t9ADqDE/EOXfIObczP90L7Aaa8qv/D3Ar4MazrxkRZMbDzG42s+fM7LmjxL0uR0RkwtSrrxSZuqHzdn64ebSNzGwJsAl42syuAw46514a75f45l1LzrntwHaAs6xxXClNRGQmUZARv3DpQTIt7ae7m3bn3OaxNjCzCuAHwLUwY10AAByaSURBVC3kbjf9D3K3lcbNN1dkRET87jC9RAlSRdjrUkQ8Z2ZBciHmO865B4HlwFLgJTN7F1gA7DSzeWPtxzdXZERE/K4134eMqVdfKXJmZsDfA7udc38N4JzbBdSP2OZdYLNzbsxLQzPiioyZ3Q88Caw2s2Yz+6TXNYmITDV1hicy7ELg48BlZvZifrh6MjuaEVdknHMf87oGEZHp1kovG4cfzBApXs65x2HsS5POuSXj2deMuCIjIjLbPcjL7KOTH/IK53InO9jldUkis4KCjIjINNvBLm7lx8PzB+nmVn6sMCMyBRRkRESm2R08QpLBY5YlSHMHj3hUkcjsoSAjIjLNWuie0HIRGT8FGRGRadZIbELLRWT8FGRERKbZVlaesCxKkNvY6kE1IrPLjHj8WkRktnI4nmE/86gkQAktdNNIjNvYyjY2eF2eiO8pyIiITKNHeIvXaeNOfofrOcvrckRmHd1aEhGZJg7H3TzOAmJcx3qvyxGZlRRkRESmydPs5zkO8GkuIEjA63JEZiUFGRGRaXI3j1NLGR9hk9eliMxaCjIiItPgVVr5NW/zKc4jStDrckRmLQUZEZFp8E0ep4IQn2CL16WIzGoKMiIiU2wvHfyE1/gEW4gR8bockVlNQUZEZIr9X35DkBI+xXlelyIy6ynIiIhMoVZ6eYCX+AibqKfC63JEZj0FGRGRKfRtniRDlk9zgdeliBQFBRkRkSnSSYJ/5DmuZT2LqPa6HJGioCAjIjJF7uMZ4qT5DBd6XYpI0VCQERGZAnFS3MvTXM4q1tLgdTkiRUNBRkRkCvwzO+kkwed4n9eliBQVBRkRkdOUIsPf8STnspjNLPS6HJGioiAjInKadvAyh+jR1RgRDyjIiIichgxZvsVvWM88LmG51+WIFB0FGRGR0/BzXmcPR/ks78Mwr8sRKToKMiIik+RwfJPHWUoNV7PW63JEipKCjIjIJP0H7/Ayh/hDLiSg/05FPKF/eSIik3Q3j9NAJR/mTK9LESlaCjIiIpOwk2ae4F1u5jzClHpdjkjRUpAREZmEb/IbYkT4Xc72uhSRoqZfI0RExmkHu7iDR2ihGwdcxWoqCHtdlkhR0xUZEZFx2MEubuXHHMyHGIBfs4cd7PK0LpFipyAjvmFVYap/8J+xKv0GLIV3B4+QIH3MsiSD3MEjHlUkIqAgIz4SuXY10W1riVyzyutSpMhkyHKQ7lHXtZxkuYgUhoKM+EbZjRvz400eVyLFIkGaf+Q5LuLuk27TSKyAFYnI8dTYV3zBqsKEzs+9VTh0wUKsMoTrTXlclcxWnST4B57lXp7mKHE20sTlrOKf2XnM7aUoQW5jq4eVioiCjPhC5NrVuFQGi5TiUhki164m8R01spTJG/kEUiMxbmMrm1nIt3mS+3mBBGm2spI/5ELOZRGGsZGmEz6zjQ1eH4pIUZsRQcbMrgLuAgLAPc65OzwuSWaYshs3UpJv5FtSFabsxk0KMnKC0cLJaEFj6AmkoasrB+nmFnaQxRGghG1s4L9xAWuoP+Zz29ig4CIyRczsXuCDQJtzbv2I5X8EfBbIAD91zt061n48DzJmFgC+CVwBNAPPmtmPnHOveVuZFFr1928g+uF1o65zA4PHzIcuXEhj9vZRt0384DU6b/j+lNc3lpEn0DlEcUA3CRqJsZWVPMJbwyfXU80P3ao4/oQ82rJtbDjpyXusk/pk103F+vFuM9FtRwsnt/JjIBdA4qQ4TB9t9HI7/3bCE0gZHBWE+BWfUbsXkcK4D7gb+IehBWZ2KXAdcJZzbsDM6k/y2WHmnDvVNtPKzM4Hvuqc+0/5+S8DOOf+8mSfOcsa3c+4uUAVSqEEVtRQ/d3rKV1ZS0lFaMKfz/alGHzzKJ0ffYDM2x3TUOHojj+Bnq4gJWRxZHBjLgtTytWs4We8zgCD41r+p1wGwF/xqwmtu5XLuJLVPMwbfH1S6y/lClYD8DBv8A0ePWGbL3Ixl7Eyf4QOBzzCm9zFfxyzbYgAN3IO57CINBnSZEmT4c95mC4SJ/x5llJClCC9DJzqjx4DDjB6QBYphCb31VGXm9nzzrnNharjrLrF7uFrbjutfcy77zOnrNnMlgA/GboiY2bfA7Y753453u+ZCUHmeuAq59yn8vMfB851zn3uZJ9RkJnFSozyPz6Xyq9dhoUCWOmpH6xzg1lcKkPv//wV/Xc9BQX+kT6XO0/6aK7MDDdyDg1U0kAFDVRyC/9KG30nbNdEjKe5xYMKRXIUZOxF4IfAVUAS+BPn3LNj7cPzW0vjZWY3Qy69NOmy7+yVdfTf+RTJn7xJ9b9cT+mqsa/O5K7CtNP5kQfI7OksYKHvKZZ+RO5iG59nx6TX/w0fAuCPePCk2/wtN2CAYRjwX/neqNsZ8DNuJkiAEAFKKeFD/D9a6T1h2yZi/AUfOGbZ/+LKE66i6QkkkfeUVESJXrT+1BuO5T7qzOy5EUu2O+e2n+JTpUANcB6wBfiemS1zY1x1mQn9yBwEFo6YX5Bfdgzn3Hbn3Gbn3OZaygpWnHgj83YH7Vu203fH47j46LdsXDxN3x2P077l256FGPC2H5EANqHlTcRO+ovAqdZ9mDNPa/1QQ9mxtvkg6/ht1nE1a/kAa0+6bSMxNjCfNdSzjFoWUc1XuIIowWO2O1k42cYGvs41NBHD8t/9da5RQ16RqdU+dN7OD6cKMZBrK/ugy3kGyAJ1Y31gJgSZZ4GVZrbUzELAR4EfeVyTzAQOMvu7cYPZ0VcPZsns6yr4raTj3cbWE06gpyNIgOBx/zRHWxYlyO9y9qgn75Mtv42to9Y7nnUw+rFOZP14t5nMthMNJ9vYwNPcwgFu52luUYgRmRn+FbgUwMxWASGgfawPeH5ryTk3aGafAx4i9/j1vc65Vz0uS2aIkY9du6zDJdJYNIiV2Ix5DHvoBOjVU0tbWDSh5UMms+74Y53o+vFuM5lth7ZXIBHxBzO7H7gEqDOzZuB24F7gXjN7BUgBvz/WbSWYAY19J0ONfYuDVYWZ1/onuU7wBgbJdibpvuXnxO68ipLqCBYuxSUHaW34hnr5FZFZYaY09t20ZI3799u/fVr7iN10UUFqngm3lkRGFbl2NS6TJdufIv18C23rv0Xye6/Stv5bpJ9vIdufwmWzRK5d7XWpIiLiEQUZmbHKbtxISXmI+D07ab/4PlxHro8Q15Gg/eL7iN+zk5KykF4iKSJSxDxvIyNyMpm2fjo+9gDJ747SZCrj6PnCQ6SeaiZy3ZrCFyciIjOCgozMWF0f+8Ept0l+99XRg46IiBQF3VoSERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER31KQEREREd9SkBERERHfUpARERER3/I0yJjZDWb2qpllzWyzl7WIiIhI4ZjZF/IZ4BUzu9/MIpPZj9dXZF4BPgQ85nEdIiIiUiBm1gT8MbDZObceCAAfncy+SqeysIlyzu0GMDMvyxAREZHCKwWiZpYGyoCWyezE6ysy42ZmN5vZc2b23FHiXpcjIiIiY6sbOm/nh5uHVjjnDgL/G9gPHAK6nXMPT+ZLpv2KjJn9Epg3yqqvOOd+ON79OOe2A9sBzrJGN0XliYiIyHGy+3rovemR091Nu3Nu1PavZlYNXAcsBbqA75vZ7znn/mmiXzLtQcY5d/l0f4eIiIj4yuXAXufcEQAzexC4AJhwkPHNrSURERGZNfYD55lZmeUaym4Fdk9mR14/fr3NzJqB84GfmtlDXtYjIiIi08859zTwALAT2EUuj2yfzL68fmppB7DDyxpERESk8JxztwO3n+5+dGtJREREfEtBRkRERHxLQUZERER8S0FGREREfEtBRkRERHxLQUZERMRjDscherwuw5c8ffxaRESkWPUxwOPs5de8zWO1Bzna101H6g5CoZDXpfmKgoyIiEgBZMiyi0P8hr08vqaDp97aRTozSEU4yiWLNrN1wzlkMhmvy/QdBRkREZFpkMXxOm08wV6e3tjLE2+8THeiD4Az+pbxmStv4IoN53HuivWESoMARKNRL0v2JQUZERGRKeBwvEU7T7CXJ9nHUxW520UAy1qa2HbOpVy09rd4/+pN1MdqPK529lCQERERmQSH423aeZJ3eYJ3eaqyhfbeLgAW1jZw5erzuHjt2bx/7SYW1DR4XO3spSAjIiIyDlkcb9DGU+zjKfbxTNUhjvR0AtBUPZfL15zL+9Zs5P1rNrG4bj65lzrLdFOQERERGUWcFC/Swk4O8DzNPF9xmI78raKFtQ1sXXUOF64+iwtXb2RZfZOCi0cUZEREpOg5HAfoygUWDvDC4m5eObCHTDb3FNGKhoVcvfLC4eCyuG6+xxXLEAUZEREpOv2keIkWdtLMTpp5oapt+DZReTjKb0XX8IWr/wvnLF/PluXrqKmIeVyxnIyCjIiIzGoZsrxFOy/QzIsc5OWFPbza/A5ZlwVyV1suX34Om5efwZZl6zhjwTJKAzo9+oX+pkREZNYY6ur/JVp4gYPsWhPnhb2v0zeQACBWVsHZVWv579d8gi3L1nH2srW62uJzCjIiIuJLWRz76ORVDrGLQ+w+I8VL+94c7rulNBBgQ3IlH7vwKs5eupbNy9axvGEBJSV6zeBsoiAjIiIzXooMb3GEV2nlFQ7x+qoku/a/TW8yDkAwUMra3qVcvel9nLV4FRsXr2LDohVEgmGPK5fppiAjIiIzShcJXuMwr9HKaxxm96J+3mh5l9RgGoCyUIT1meV85PwrOXPxKs5ctJJ1TUsJB/WyxWKkICMiIp7IkGUfnbxGK7s5zG7aeL22m/1HW4e3mVtVzYbKFVx6+fWctWglZy5eyfKGBQRKAh5WLjOJgoyIiEwrh6OVXt7kCG9xhDdo482lSXYf3Es8lQQgUBJg5byFbFlwBjddeh0bFq5gw6IVNMRqPa5eZjoFGRERmRJZHC108yZHeJt23uIIe5aneKNl3/BbnwFqKmKcEV7GH1x8DWcsWM76RctZ07hE7VlkUhRkRERkQlJkeJcO3qadPbTzFu3sWZzkrUP7h6+wANRWxFhbupTrz7uctY1LWNO0lDWNS5hbVe1h9TLbKMiIiMioOoizh3b2cHR4/E5DnL1HWoa77ofcCxNXVSzm9y/6IKsaF7OmcQmr5y+mtnKOh9VLsVCQEREpYgnS7KOTdzg6PLy7PM1brQfo7O8Z3i5UGmR5wwLWzVvG72y5hNXzF7Ny/mJWNCykMlrm4RFIsVOQERGZ5XpIsp9O3qWTfXSwn06a1xrvtDXT3NGGc25424ZYDStLF3Hd5otZNX8xK+ctZMW8RSyum6cnhWRGUpAREfG5QbIcoof9dLKPTg7QyX66aF6aYe+RFjryPd0Oqa2IsXSgiQtWncWy+iZWzFvEinkLWFa/gKpouUdHITI5CjIiIjNcFkcbfTTTxX46OUAXB+iiZa2x78ghDna2MZh5r81KaSDAgpoGFkfmc+3ZF7G0vomlcxtZWt/EkrmNCisyqyjIiIh4LE2GVno5SDfNdHGQbg7QReu6APvbD9Hc0Tbcq+2QhlgNi1Lz2bL8DD5ct5UlcxtZMnc+S+Y20lQ9V29vlqKhn3QRkWmUxdFBnFZ6aKGHFrppoYeDdHN4hXHgaCuHuo6SddljPtcQq2Fhch4bl6zmmrMvYmHtPJbUzWdR3TwW1c0nGlKfKyKgICMiMmkJ0hyml8P00kovrfTkx720rwzQ0nmEQ13tJ1xNCQZKaaqpZ2FpAxevO5sFNQ0srG1gQW0DC2saWFjXoM7hRMZJQUZEZASHo58UbfRxmF7a6OPIiOn2tUFau9pp7Tp6TG+1Q8pCEeZX1zG/pI5zVpxBY/VcmqrraayeS2N1HQtqG5hbWU1JSYkHRycy+yjIiEhRSJDmCH0coZ+j9Oenc0Mb/XSsCNDW00lbdwf9A4kTPh8qDTIvVsu8VC2rG5dw8bqzmTenjvlzammI1dJYPZf51XXEohWYmQdHKFKcFGRExJdSZDhKP53EOUqc9nxAyQ3x3LrlpbT1dNLe00nfKOEEoLq8ioZYDfXBGjYvW0t9VQ0NsVrm5QNKQyw3X11eqYAiMgP5MsjspYMv8kOqiVJNlDlEmUPZiOncUEYQQ//xiMxkQ7dyuknSRYIuEnSSoIt4fpyb7yBO17JSjvZ1c7S3i55E/6j7C5QEqKuMMbeymtpQBVuWr6O+qoa6yjnUV9VQH6umrrKa+lg19VU1hEqDBT5iEZlKvgwygyF4oqKFjr6eY15QdrzSQIDq8iqqyyuZU1ZJ+Z4BYkSJEWEOEWJEqSJCFRFi+fHQUEmYUnQPW2Q8BsnSS5JukvQyQE9+OjdO0EOSrvyybhL0LQ/T1d9LV7yXzv6eY/pAOV5ZKEJNRRU1FTFqImUsmdtIXWWMuso51FTMoa4yRm3lHOqrqqmrnMOcskq1PxEpIp4GGTP7BnANkAL2ADc657pO9bnV85fw77d/G4BkeoCOvh46+3vo6Ouhqz/3H2NnftwV76WzL/8f5uJB9saP0hXvpTved0y33KMpD0epipbnhrIKyvYkqSQ8PFQcNy4nTAWh4XEFYcoJEfZnXpRZzuEYIEOcFH0M0EeK/hHjXgboO248NPQvC9OT6KM3Eacn0T9qm5KRzIxYWQVzynK/VMwJR2mqrqe6opLq8irmlFVSXZ6fLq/MBZfyGNUVlXp6R2SWMrOrgLuAAHCPc+6OyezH6zPsL4AvO+cGzeyvgC8DfzqRHUSC4fzTAHMn9MXZbJaeZD9d/b30JPrpSfTTnQ84PfF+uhN99MT76M6v60n00bM0S0uij75kG72J/pPecz9eMFBKeSRKRThKeThKeSRKZG+SckKUESJKkDKClBEaHkdHjI8fIpQSGTHWlaPZa5AsSdIkGRweJ0ifMMRJDY/jI8ZDy/pJkVwaoT+ZoH8gQd9Agv5kgnRmcFx1VISjVEbLqYyWURWtoDJaRlNNPVXRcmL5oB+LVhArq6CqrJxYWSWx6NB0BVWRcl0lEZFhZhYAvglcATQDz5rZj5xzr010X54GGefcwyNmnwKuL9R3l5SUDP92OFmZbIa+ZILeZD+9iTi9iX56k3H6knH6kgn6kvHh+f78iWPoBBJfFaI1mSA+0EV8IJlbP5Akkz35JfaTKQ0EiAbDREJhosEw4WCISDBEOBii9J1+wpQODyEC+aGUcH48tCyYH0KUEqRkeL40Pz1yXDo8fm8IDA92zHTJiGWG5ZdZfm762zA5HNn8kMHhcGTIksGRzY9z88dODx4zZIan0/npoXGKDINkSJMlxSBpMqTJMJAfpxhkID9OkWFgxHiAQQaXlTOQTpFMpxhIp0ikB0imBkikB8a85XIygZIAFeEoZeFIfohSGamiNhxlUe18ysMRyiO5UF0RKaMyUkZFpIyKyHvzQ6GlMlJORSSqlwWKyFQ7B3jbOfcOgJn9C3Ad4K8gc5ybgO96XcREBEoCxMpyv4VOBeccqcE0/QNJ4qkEyVSKeCpJIpWkfyA3TgwMHHOiS6Ryw9B8ajBNMp0imRrInRhXBelLp+gYTJNMJxgYTJFKpxkYTDOQTpHKpCd1spxKJVaCmVFihlku3Jgx/ITIUNgZ+cTI0G1Bhxuedy4fWlw2N+3cCb2lFlppIEAoECQcDBEuDRIKBgmX5oJmqDREJFhBWTBENFYzHEBDpcHhYFoWihAJhY4JqmXhCNFQhLJwODfOb1MWilIejhAqDerpGhGZ6ZqAAyPmm4FzJ7OjaQ8yZvZLYN4oq77inPthfpuvAIPAd8bYz83AzfnZgdhNF70y1bX6SB3Q7nURUyXrsuBgnHHKV8c+mMkwmMmM2Sh9gnx1/NNAx6/jn93Hf9NJ16wuYBW8zKGHFvBndae5m4iZPTdifrtzbvtp7vME0x5knHOXj7XezP4A+CCw1Y3R+jZ/8Nvzn3nOObd5Kuv0k2I+/mI+dtDx6/h1/MV6/McFgmnnnLtqmr/iILBwxPyC/LIJ87T1Xb7F8q3Atc65uJe1iIiISME8C6w0s6VmFgI+CvxoMjvyuo3M3UAY+EX+nv5TzrlPe1uSiIiITKf808qfAx4i9/j1vc65VyezL6+fWloxyY9O+T02nynm4y/mYwcdv46/uBXz8c+6Y3fO/Qz42enux07VKZyIiIjITKUeqkRERMS3fBtkzOwbZva6mb1sZjvMbI7XNRWKmd1gZq+aWdbMiqYFv5ldZWZvmNnbZnab1/UUkpnda2ZtZlaU3Q6Y2UIze9TMXsv/7H/e65oKxcwiZvaMmb2UP/Y/87omL5hZwMxeMLOfeF1LoZnZu2a2y8xeLPTTS37g2yBD7vUG651zZwJvknu9QbF4BfgQ8JjXhRTKiO6sPwCsAz5mZuu8raqg7gOm+3HImWwQ+JJzbh1wHvDZIvr7HwAuc86dBWwErjKz8zyuyQufB3Z7XYSHLnXObSzWx8/H4tsg45x72Dk39KKYp8g9g14UnHO7nXNveF1HgQ13Z+2cSwFD3VkXBefcY0CH13V4xTl3yDm3Mz/dS+6E1uRtVYXhcvrys8H8UFSNG81sAfDbwD1e1yIzj2+DzHFuAv7N6yJkWo3WnXVRnMjkWGa2BNgEPO1tJYWTv63yItAG/MI5VzTHnncnuT7HvH3niHcc8LCZPZ/v5V5G8LofmTFN1esN/Gg8xy5SbMysAvgBcItzrsfregrFOZcBNubbAu4ws/XOuaJoL2VmHwTanHPPm9klXtfjkfc55w6aWT25ftdez1+lFWZ4kJmq1xv40amOvQhNWXfW4k9mFiQXYr7jnHvQ63q84JzrMrNHybWXKoogA1wIXGtmVwMRoMrM/sk593se11UwzrmD+XGbme0gd6tdQSbPt7eW9HqDojNl3VmL/1iu6++/B3Y75/7a63oKyczmDj2VaWZR4ArgdW+rKhzn3Jedcwucc0vI/bv/VTGFGDMrN7PKoWngSoonxI6Lb4MMudcbVJK7zPaimf2t1wUVipltM7Nm4Hzgp2b2kNc1Tbd8w+6h7qx3A9+bbHfWfmRm9wNPAqvNrNnMPul1TQV2IfBx4LL8v/cX87+hF4P5wKNm9jK5QP8L51zRPYJcxBqAx83sJeAZ4KfOuZ97XNOMop59RURExLf8fEVGREREipyCjIiIiPiWgoyIiIj4loKMiIiI+JaCjIiIiPiWgoyIiIj4loKMiIiI+JaCjIhMiJnda2ZtZqbeRUXEcwoyIjJR95F714+IiOcUZERkVGb2qJldkZ/+mpn9DUD+rbsdnhYnIpI3o99+LSKeuh34czOrBzYB13pcj4jICRRkRGRUzrnH8m+d/iJwiXMu43VNIiLH060lERmVmW0g9+bllHOu1+t6RERGoyAjIicws/nAd4DrgD4zU+NeEZmRFGRE5BhmVgY8CHzJObcb+Aty7WWG1t8PPAmsNrNmM/ukN5WKiIA557yuQURERGRSdEVGREREfEtBRkRERHxLQUZERER8S0FGREREfEtBRkRERHxLQUZERER8S0FGREREfEtBRkRERHzr/wOJJuLOuYEYqQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from nndl.op import SimpleBatchGD\n",
    "\n",
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "w = paddle.to_tensor([0.2, 2])\n",
    "model = OptimizedFunction(w)\n",
    "opt = SimpleBatchGD(init_lr=0.2, model=model)\n",
    "train_and_plot_f(model, opt, epoch=20, fig_name='opti-vis-para.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "输出图中不同颜色代表$f(x_1, x_2)$的值，具体数值可以参考图右侧的对应表，比如深粉色区域代表$f(x_1, x_2)$在0～8之间，不同颜色间黑色的曲线是等值线，代表落在该线上的点对应的$f(x_1, x_2)$的值都相同。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.3.1.2 简单拟合实验\n",
    "\n",
    "除了2D可视化实验外，我们还设计一个简单的拟合任务，然后对比不同的优化算法。\n",
    "\n",
    "这里我们随机生成一组数据作为数据样本，再构建一个简单的单层前馈神经网络，用于前向计算。\n",
    "\n",
    "**数据集构建**  通过paddle.randn随机生成一些训练数据$\\bm X$，并根据一个预定义函数$y = 0.5\\times x_{1}+ 0.8\\times x_{2} + 0.01\\times noise$ 计算得到$\\bm y$，再将$\\bm X$和$\\bm y$拼接起来得到训练样本。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  [-4.080414  -1.3719953]\n",
      "y:  [-3.136211]\n",
      "input data shape:  [1000, 3]\n",
      "data:  [-4.080414  -1.3719953 -3.136211 ]\n"
     ]
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "# 随机生成shape为（1000，2）的训练数据\n",
    "X = paddle.randn([1000, 2])\n",
    "w = paddle.to_tensor([0.5, 0.8])\n",
    "w = paddle.unsqueeze(w, axis=1)\n",
    "noise = 0.01 * paddle.rand([1000])\n",
    "noise = paddle.unsqueeze(noise, axis=1)\n",
    "# 计算y\n",
    "y = paddle.matmul(X, w) + noise\n",
    "# 打印X, y样本\n",
    "print('X: ', X[0].numpy())\n",
    "print('y: ', y[0].numpy())\n",
    "\n",
    "# X，y组成训练样本数据\n",
    "data = paddle.concat((X, y), axis=1)\n",
    "print('input data shape: ', data.shape)\n",
    "print('data: ', data[0].numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**模型构建**  定义单层前馈神经网络，$\\bm X \\in \\mathbb{R}^{N \\times D}$为网络输入, $\\bm w \\in \\mathbb{R}^{D}$是网络的权重矩阵，$\\bm b \\in \\mathbb{R}$为偏置。\n",
    "$$\n",
    "\\bm y =\\bm X \\bm w + b \\in \\mathbb{R}^{K\\times 1},\n",
    "$$\n",
    "\n",
    "其中$K$代表一个批次中的样本数量，$D$为单层网络的输入特征维度。\n",
    "\n",
    "**损失函数**  使用均方误差作为训练时的损失函数，计算损失函数关于参数$\\bm w$和$b$的偏导数。定义均方误差损失函数的计算方法为\n",
    "$$\n",
    "\\mathcal{L} = \\frac{1}{2K}\\sum_{k=1}^K(\\bm y^{(k)} - \\bm z^{(k)})^2,\n",
    "$$\n",
    "\n",
    "其中$\\bm z^{(k)}$是网络对第$k$个样本的预测值。根据损失函数关于参数的偏导公式，得到$\\mathcal{L}(\\cdot)$对于参数$\\bm w$和$b$的偏导数，\n",
    "$$\n",
    "    \\frac{\\partial \\mathcal{L}}{\\partial \\bm w} = \\frac{1}{K}\\sum_{k=1}^K\\bm x^{(k)}(\\bm z^{(k)} - \\bm y^{(k)}) = \\frac{1}{K}\\bm X^\\top(\\bm z - \\bm y), \\\\\n",
    "    \\frac{\\partial \\mathcal{L}}{\\partial b} = \\frac{1}{K}\\sum_{k=1}^K(\\bm z^{(k)} - \\bm y^{(k)}) = \\frac{1}{K}\\mathbf{1}^\\top(\\bm z - \\bm y).\n",
    "$$\n",
    "\n",
    "定义Linear算子，实现一个线性层的前向和反向计算。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Linear(Op):\n",
    "    def __init__(self, input_size, weight_init=paddle.standard_normal, bias_init=paddle.zeros):\n",
    "        super(Linear, self).__init__()\n",
    "        self.params = {}\n",
    "        self.params['W'] = weight_init(shape=[input_size, 1])\n",
    "        self.params['b'] = bias_init(shape=[1])\n",
    "        self.inputs = None\n",
    "        self.grads = {}\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        self.inputs = inputs\n",
    "        self.outputs = paddle.matmul(self.inputs, self.params['W']) + self.params['b']\n",
    "        return self.outputs\n",
    "\n",
    "    def backward(self, labels):\n",
    "        K = self.inputs.shape[0]\n",
    "        self.grads['W'] = 1. /K * paddle.matmul(self.inputs.T, (self.outputs - labels))\n",
    "        self.grads['b'] = 1. /K * paddle.sum(self.outputs - labels, axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**笔记**\n",
    "\n",
    "这里backward函数中实现的梯度并不是forward函数对应的梯度，而是最终损失关于参数的梯度．由于这里的梯度是手动计算的，所以直接给出了最终的梯度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**训练函数**  在准备好样本数据和网络以后，复用优化器SimpleBatchGD类，使用小批量梯度下降来进行简单的拟合实验。\n",
    "\n",
    "这里我们重新定义模型训练train函数。主要以下两点原因：\n",
    "* 在一般的随机梯度下降中要在每回合迭代开始之前随机打乱训练数据的顺序，再按批大小进行分组。这里为了保证每次运行结果一致以便更好地对比不同的优化算法，这里不再随机打乱数据。\n",
    "* 与RunnerV2中的训练函数相比，这里使用小批量梯度下降。而与RunnerV3中的训练函数相比，又通过继承优化器基类Optimizer实现不同的优化器。\n",
    "\n",
    "模型训练train函数的代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def train(data, num_epochs, batch_size, model, calculate_loss, optimizer, verbose=False):\n",
    "    \"\"\"\n",
    "    训练神经网络\n",
    "    输入：\n",
    "        - data：训练样本\n",
    "        - num_epochs：训练回合数\n",
    "        - batch_size：批大小\n",
    "        - model：实例化的模型\n",
    "        - calculate_loss：损失函数\n",
    "        - optimizer：优化器\n",
    "        - verbose：日志显示，默认为False\n",
    "    输出：\n",
    "        - iter_loss：每一次迭代的损失值\n",
    "        - epoch_loss：每个回合的平均损失值\n",
    "    \"\"\"\n",
    "    # 记录每个回合损失的变化\n",
    "    epoch_loss = []\n",
    "    # 记录每次迭代损失的变化\n",
    "    iter_loss = []\n",
    "    N = len(data)\n",
    "    for epoch_id in range(num_epochs):\n",
    "        # np.random.shuffle(data) #不再随机打乱数据\n",
    "        # 将训练数据进行拆分，每个mini_batch包含batch_size条的数据\n",
    "        mini_batches = [data[i:i+batch_size] for i in range(0, N, batch_size)]\n",
    "        for iter_id, mini_batch in enumerate(mini_batches):\n",
    "            # data中前两个分量为X\n",
    "            inputs = mini_batch[:, :-1]\n",
    "            # data中最后一个分量为y\n",
    "            labels = mini_batch[:, -1:]\n",
    "            # 前向计算\n",
    "            outputs = model(inputs)\n",
    "            # 计算损失\n",
    "            loss = calculate_loss(outputs, labels).numpy()[0]\n",
    "            # 计算梯度\n",
    "            model.backward(labels)\n",
    "            # 梯度更新\n",
    "            optimizer.step()\n",
    "            iter_loss.append(loss)\n",
    "        # verbose = True 则打印当前回合的损失\n",
    "        if verbose:\n",
    "            print('Epoch {:3d}, loss = {:.4f}'.format(epoch_id, np.mean(iter_loss)))\n",
    "        epoch_loss.append(np.mean(iter_loss))\n",
    "    return iter_loss, epoch_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**优化过程可视化**  定义plot_loss函数，用于绘制损失函数变化趋势。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def plot_loss(iter_loss, epoch_loss, fig_name):\n",
    "    \"\"\"\n",
    "    可视化损失函数的变化趋势\n",
    "    \"\"\"\n",
    "    plt.figure(figsize=(10, 4))\n",
    "    ax1 = plt.subplot(121)\n",
    "    ax1.plot(iter_loss, color='#e4007f')\n",
    "    plt.title('iteration loss')\n",
    "    ax2 = plt.subplot(122)\n",
    "    ax2.plot(epoch_loss, color='#f19ec2')\n",
    "    plt.title('epoch loss')\n",
    "    plt.savefig(fig_name)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "对于使用不同优化器的模型训练，保存每一个回合损失的更新情况，并绘制出损失函数的变化趋势，以此验证模型是否收敛。定义train_and_plot函数，调用train和plot_loss函数，训练并展示每个回合和每次迭代(Iteration)的损失变化情况。在模型训练时，使用paddle.nn.MSELoss()计算均方误差。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import paddle.nn as nn\n",
    "def train_and_plot(optimizer, fig_name):\n",
    "    \"\"\"\n",
    "    训练网络并画出损失函数的变化趋势\n",
    "    输入：\n",
    "        - optimizer：优化器\n",
    "    \"\"\"\n",
    "    # 定义均方差损失\n",
    "    mse = nn.MSELoss()\n",
    "    iter_loss, epoch_loss = train(data, num_epochs=30, batch_size=64, model=model, calculate_loss=mse, optimizer=optimizer)\n",
    "    plot_loss(iter_loss, epoch_loss, fig_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "训练网络并可视化损失函数的变化趋势。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAEICAYAAAB74HFBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcnWV9///X52yzZyaTTCY7IQQIixIwbG4FBARqRa21+LNu1VJbtWK1Vu33q6htXVr1a6XVouJexcoiKiiIWKUqEEICgUASQvbJZLLOPmf7/P6475mcTM7JnGSWs8z7+Xicx7nv677ucz5nMrnnc67ruq/L3B0REREROTGRUgcgIiIiUsmUTImIiIiMg5IpERERkXFQMiUiIiIyDkqmRERERMZByZSIiIjIOCiZqjJm9qSZXVLC919sZr1mFp2E13YzWzbRrysiMpqZLQmvObEi63/DzP5xsuOS8qRkqsq4+1nu/isAM7vRzL4zme9nZlvM7PKc99/m7o3unpnM9xURESkXSqakoGK/kYmIiExnSqaqzHBLkZldBXwY+NOw221teLzZzL5mZh1mttPM/nG4S87M3mJm/2tmnzezfcCNZnaKmf3SzPaZ2V4z+66ZtYT1vw0sBn4cvscHRjeNm9l8M7vLzPab2SYz+4ucWG80sx+Y2bfMrCfsolxZ5OdsDs/rMrOtZvZ/zCwSHltmZv9jZofCmG8Nyy38bHvMrNvMnjCzsyfshy8ikya8ltwW/p9/zsz+JufYjWb2QzO7NbyWrDazc3KOn2FmvzKzg+F15pU5x+rM7LPhdeSQmT1oZnU5b/0GM9sWXkv+4Tji/Yvwmrc/vAbOD8sLXofM7Bozeyr8DDvN7P3j+qHJlFEyVaXc/WfAPwO3ht1uwxeWbwBpYBlwLnAl8PacUy8ENgPtwD8BBnwSmA+cASwCbgzf443ANuCPwvf4TJ5Qvg/sCM9/LfDPZnZZzvFXhnVagLuAm4r8iF8EmoGlwB8AbwLeGh77BHAvMBNYGNYl/KwvBU4Lz30dsK/I9xOREgm/KP0YWAssAF4G3GBmL8+pdi3w30Ar8F/AnWYWN7N4eO69wBzg3cB3zez08Lx/BV4AvDA89wNANud1XwycHr7nR8zsjCLivYzguvk6YB6wleA6B8e+Dn0N+Et3bwLOBn451ntJeVAyNY2YWTtwDXCDu/e5+x7g88B1OdV2ufsX3T3t7gPuvsnd73P3IXfvAj5HkLwU836LgBcBf+/ug+6+BvgqQeIz7EF3vzscY/Vt4Jw8LzX6daNhzB9y9x533wJ8FnhjWCUFnATMD9/3wZzyJmA5YO6+3t07ivksIlJS5wNt7v5xd0+6+2bgKxx57XrU3X/o7imC61QtcFH4aAQ+FZ77S+AnwOvDJO3Pgfe4+053z7j7b919KOd1PxZeC9cSJHNjXqOANwC3uPvq8LU+BFxsZks49nUoBZxpZjPc/YC7rz7un5SUhJKp6eUkIA50hM3dB4H/JPi2Nmx77glm1m5m3w+bnLuB7wCzi3y/+cB+d+/JKdtK8M1y2O6c7X6gtoixWrPDz7G1wOt+gKBF7eGwSf/PAcKL6E3AvwN7zOxmM5tR5GcRkdI5CZg/fN0Kr10fJmhBHzZy7XL3LIdbxOcD28OyYcPXi9kESdezx3jv0deoxiLinU/O9cndewlanxaMcR36Y4IvvFvDoQoXF/FeUgaUTFU3H7W/HRgCZrt7S/iY4e5nHeOcfw7LnufuM4A/I0hUCtXPtQtoNbOmnLLFwM7j+RB57OVw69NRr+vuu939L9x9PvCXwH9YOKWCu/+bu78AOJOgmf3vxhmLiEy+7cBzOdetFndvcvdrcuosGt4IW5wWElyDdgGLhsdUhoavF3uBQeCUCY53FznXJzNrAGZx+BqV9zrk7o+4+7UEX3DvBH4wwXHJJFEyVd06gSXDF5GwKfle4LNmNsPMIhYMMD9Wt10T0AscMrMFHJ18dBKMWzqKu28Hfgt80sxqzez5wNsIWrdOWNgl+APgn8ysycxOAv52+HXN7E/MbGFY/QBBwpc1s/PN7MJwDEUfwUU0e/Q7iEiZeRjoMbO/DweMR83sbDM7P6fOC8zsNWHL9g0EXxx/DzxE0KL0gXAM1SXAHwHfD1urbgE+Fw5wj5rZxWZWM854vwe81cxWhK/1z8BD7r6l0HXIzBJm9gYzaw67KrvR9aliKJmqbv8dPu8zs+G+9zcBCeApgkTjhwQDJAv5GHAecAj4KXD7qOOfBP5P2PSe786T1wNLCL6p3QF81N1/cfwf5SjvJrgQbQYeJBhwekt47HzgITPrJRjU/p5wjMUMgnEWBwia4PcB/zIBsYjIJAq/QL0CWAE8R9Ci9FWCAdzDfgT8KcH/7zcCr3H3lLsnCZKnq8Pz/gN4k7s/HZ73fuAJ4BFgP/Bpxvm3MbzG/V/gNqCDoOVreHzXsa5DbwS2hEMq3kEw9koqgLkfq5dGRESkvJnZjcAyd/+zUsci05NapkRERETGQcmUiIiIyDiom09ERERkHNQyJSIiIjIOU7qQ7ezZs33JkiVT+ZYiUmKPPvroXndvK3Uc46Xrl8j0U+z1a0qTqSVLlrBq1aqpfEsRKTEz2zp2rfKn65fI9FPs9UvdfCIiIiLjoGRKREREZByUTImIiIiMw5jJVLim2sNmttbMnjSzj4Xl3zCz58xsTfhYMfnhioiIiJSXYgagDwGXuXtvuDDjg2Z2T3js79z9h5MXnoiIiEh5GzOZ8mBWz95wNx4+NNOniIiICEWOmTKzqJmtAfYA97n7Q+GhfzKzx83s82ZWU+Dc681slZmt6urqmqCwRURERMpDUcmUu2fcfQWwELjAzM4GPgQsB84HWoG/L3Duze6+0t1XtrUVN29fetM+uj/yS9LbDhZVX0SkXHjWGVq3g9T2faUORUSmyHHdzefuB4EHgKvcvcMDQ8DXgQsmKqj05gP0fOLXZHZ0T9RLiohMCYsYyfW7SG3oLHUoIjJFirmbr83MWsLtOuAK4GkzmxeWGfAqYN3ERWXBc1ZDs0Sk8sTmtpDpPIQWkheZHoq5m28e8E0zixIkXz9w95+Y2S/NrA0wYA3wjgmLypRMiUjlis5tJrWpk+yBfqKtDaUOR0QmWTF38z0OnJun/LJJiYigmTx4k8l6BxGpdGa2CPgW0E5wtbjZ3b9gZv8C/BGQBJ4F3hoOURh9/hagB8gAaXdfOVGxxeY2A5DZfUjJlMg0UJ4zoIe5lFqmROQY0sD73P1M4CLgnWZ2JnAfcLa7Px/YQHCzTCGXuvuKiUykACJNtVhDDenOQxP5siJSpsozmRppmVIyJSL5hTfBrA63e4D1wAJ3v9fd02G13xPchTzlYnObyezWuCmR6aCskylXy5SIFMHMlhAMR3ho1KE/B+4ZXT/kwL1m9qiZXV/gdU94nrzo3GZ8MEX2UP9xnScilac8kykNQBeRIplZI3AbcIO7d+eU/wNBV+B3C5z6Ync/D7iaoIvwpaMrnMg8ecNi88JxUx3q6hOpduWZTGkAuogUIVwv9Dbgu+5+e075W4BXAG/wAv1s7r4zfN4D3MEEzpUHYI21WH2C9G4lUyLVriyTKdMAdBEZQzjH3deA9e7+uZzyq4APAK9097x9bGbWYGZNw9vAlUzkXHnB62rclMg0UZbJlAagi0gRXgS8EbjMzNaEj2uAm4Am4L6w7MsAZjbfzO4Oz20HHjSztcDDwE/d/WcTHWB0Xks4bmpgol9aRMpIMZN2Tj0NQBeRMbj7gxyeSCXX3XnKcPddwDXh9mbgnMmLLhBtz5lvqqV+st9OREqkPFumNABdRKpAZIbGTYlMB+WZTGkAuohUATMjqnFTIlWvPJMpDUAXkSoRm9uMDyTJdmvclEi1KstkyjQAXUSqRDRnnT4RqU5lmUyNdPOpZUpEKlxkRh1WF9e4KZEqVp7JlOluPhGpDho3JVL9yjOZ0gB0EakisbkteH8S7xksdSgiMgnKM5nSAHQRqSLD46bU1SdSncozmdIAdBGpIpHmOqw2rkHoIlWqLJMp0wB0Eakiw+Om0ho3JVKVyjKZ0gzoIlJtYnOb8b4hvHeo1KGIyAQrz2RqeG0+5VIiUiUOj5s6WOJIRGSijZlMmVmtmT1sZmvN7Ekz+1hYfrKZPWRmm8zsVjNLTFhUGoAuIlUm0lKP1cQ0bkqkChXTMjUEXObu5wArgKvM7CLg08Dn3X0ZcAB428RFpQHoIlJdcsdNiUh1GTOZ8kBvuBsPHw5cBvwwLP8m8KqJCkoD0EWkGsXmNuO9Q2Q135RIVSlqzJSZRc1sDbAHuA94Fjjo7umwyg5gQYFzrzezVWa2qqurq7ioNABdRMZgZovM7AEzeyocgvCesLzVzO4zs43h88wC5785rLPRzN48FTFH57YAmm9KpNoUlUy5e8bdVwALgQuA5cW+gbvf7O4r3X1lW1tbkVFpBnQRGVMaeJ+7nwlcBLzTzM4EPgjc7+6nAveH+0cws1bgo8CFBNe0jxZKuiZSZGY4bqpTyZRINTmuu/nc/SDwAHAx0GJmsfDQQmDnhEU1nEupZUpECnD3DndfHW73AOsJWsivJRh6AIWHILwcuM/d97v7AYIW96smO2YzI9qucVMi1aaYu/nazKwl3K4DriC4aD0AvDas9mbgRxMXlQagi0jxzGwJcC7wENDu7h3hod1Ae55TFgDbc/YLDlWYaNG5zXjPIFnNNyVSNYppmZoHPGBmjwOPEHyb+wnw98DfmtkmYBbwtYmLSmOmRKQ4ZtYI3Abc4O7ducc8mG78hC8kJzTmcwyxeeF8U+rqE6kasbEquPvjBN/4RpdvJhhrMPE0AF1EimBmcYJE6rvufntY3Glm89y9w8zmEdw4M9pO4JKc/YXAr0ZXcvebgZsBVq5cOSEXpMjMhmDc1M4DcMqciXhJESmxspwB3TQAXUTGYGZG0CK+3t0/l3PoLoKhB1B4CMLPgSvNbGY48PzKsGzSmRmxha2kd+zXuFCRKlGWyZRmQBeRIrwIeCNwmZmtCR/XAJ8CrjCzjcDl4T5mttLMvgrg7vuBTxAMXXgE+HhYNiVii2fhQ2kye7rHriwiZW/Mbr6SGFmbT8mUiOTn7g9y+KvXaC/LU38V8Pac/VuAWyYnumOLLWiBiJHeto9YuGafiFSu8myZ0gB0EaliFo8Rm9dCats+fWkUqQLlmUxpALqIVLnY4lnBFAmH+ksdioiMU1kmUxqALiLVLraoFYD0tikbqiUik6QskykNQBeRahdpqCEyu5H0tn2lDkVExqk8kynNgC4i00B80SwyXT1k+5OlDkVExqGskynNwSIi1Sy2eBYA6R3q6hOpZOWZTGkAuohMA5GZ9Vhjjbr6RCpceSZTGoAuItOAmRFfNIv0roN4KlPqcETkBJVlMmVhy1TPjb+i+//+ssTRiIhMntjiVshkSe86WOpQROQElWUyBYzc0dfzj78ubRwiIpMoOrcZElHS29XVJ1KpyjeZihRaJUJEpHpYJBIsfLxdCx+LVKryTabscDLlmayWXBCRqhVfNAsfTJHp6il1KCJyAso3mcppmdoV+zj7r/1eCYMREZk8sYUzRxY+FpHKU77J1KhevsEfbyhNHCIik8wSMaJzmzVuSqRClW0yZRozJSLTSHzRLLKHBsho4WORilO2yZQGoIvIdBJbrIWPRSrVmMmUmS0yswfM7Ckze9LM3hOW32hmO81sTfi4ZkIjs6OTqZ5P/WZC30JEKpeZ3WJme8xsXU7ZrTnXpC1mtqbAuVvM7Imw3qqpi7qwSGMtkdYGdfWJVKBiWqbSwPvc/UzgIuCdZnZmeOzz7r4ifNw9oZHlaZjq/tD9E/oWIlLRvgFclVvg7n86fE0CbgNuP8b5l4Z1V05ijMcltngWmT3dZAe18LFIJRkzmXL3DndfHW73AOuBBZMdmJaSEZFjcfdfA3n7xCxYRuF1QEXdBhxfNAsc0tsPlDoUETkOxzVmysyWAOcCD4VF7zKzx8Pm9pkFzrnezFaZ2aqurq6i38vT2eMJTUQk10uATnffWOC4A/ea2aNmdn2hFznR69eJisxqwOoTmiJBpMIUnUyZWSNBs/kN7t4NfAk4BVgBdACfzXeeu9/s7ivdfWVbW1vxkWUKJ1M9n/oN+175X8W/lohMN6/n2K1SL3b384CrCYYuvDRfpRO+fp0gMyO2eBbpXQfwtBY+FqkURSVTZhYnSKS+6+63A7h7p7tn3D0LfAW4YEIjy+Tv53N3uj90v+adEpG8zCwGvAa4tVAdd98ZPu8B7mCir1/jEF88C9JZ0h2HSh2KiBSpmLv5DPgasN7dP5dTPi+n2quBdaPPHZcCa1R5z9CEvo2IVJ3LgafdfUe+g2bWYGZNw9vAlUz09WscRhY+fm7yuxVFZGIU0zL1IuCNwGWjpkH4THhr8ePApcB7JzPQERpLJSKAmX0P+B1wupntMLO3hYeuY1QXn5nNN7PhO47bgQfNbC3wMPBTd//ZVMU9FotGiJ/cRmrLXjyZLnU4IlKE2FgV3P1B8k5UwMROhVAkTymZEhFw99cXKH9LnrJdwDXh9mbgnEkNbpzip7aTemY3qS17SZw2t9ThiMgYyncG9AK8W918IlLdorObiDTXkdrUWepQRKQIlZdMDaRKHYKIyKQyM+LL2sl0dpPtHih1OCIyhspLpvqVTIlI9YsvmwMGyY1qnRIpd0qmRETKUKS+htj8maQ2deIF7m4WkfJQcclUNieZctcFRkSqV/zUdrw/SWb3wVKHIiLHUHHJ1BEtU/q2JiJVLLZoFiRi6uoTKXOVnUwdY8kZEZFKZ7EI8aVtpLfu05xTImWs8pKpgZwLilqmRKTKJZa1QyZLSjOii5StykumcsdMKZkSkSoXmd1IpKWelLr6RMpWxSVT3e+/9/COkikRqXIjc0519ZA51F/qcEQkj4pLpo6QUTIlItUvfkow55Rap0TKU2UnU2qZEpFpIFKfILawldSzezS8QaQMVXQypYuKiEwX8WXhnFMdmnNKpNxUdDKllikRmS5ii1qxmhjJjbtLHYqIjFLZyZTmmRKRacKiEeJL55Detg8f0rJaIuWkYpKppo/8wdGFapkSmbbM7BYz22Nm63LKbjSznWa2JnxcU+Dcq8zsGTPbZGYfnLqoxye+rB0yTuq5vaUORURyVEwyFV8x9+hCJVMi09k3gKvylH/e3VeEj7tHHzSzKPDvwNXAmcDrzezMSY10gkRmNRCZ2aDlZUTKTNkmU3N3vY+5e/6O6NKZAFhd7Kg6GoAuMn25+6+B/Sdw6gXAJnff7O5J4PvAtRMa3CQxM+KntpPd20PmYF+pwxGRUNkmU9F5TUTbGiBiAFh9/OhKmmdKRI72LjN7POwGnJnn+AJge87+jrDsKGZ2vZmtMrNVXV3lsZxLfGkbRIzk0x2lDkVEQmWbTA2zaJhM1eVJptQyJSJH+hJwCrAC6AA+O54Xc/eb3X2lu69sa2ubiPjGLVKXIL50DqmNnWQHNRBdpByMmUyZ2SIze8DMnjKzJ83sPWF5q5ndZ2Ybw+d83wDHLxqGGM8TqpIpEcnh7p3unnH3LPAVgi690XYCi3L2F4ZlFSNx9gJIZ0mpdUqkLBTTMpUG3ufuZwIXAe8MB2t+ELjf3U8F7g/3J95wy5TZUYc0ZkpEcpnZvJzdVwPr8lR7BDjVzE42swRwHXDXVMQ3UaIzG4gtbCW5fheezpQ6HJFpb8xkyt073H11uN0DrCcYX3At8M2w2jeBV01GgBa2THm+OaU0z5TItGVm3wN+B5xuZjvM7G3AZ8zsCTN7HLgUeG9Yd76Z3Q3g7mngXcDPCa5nP3D3J0vyIcYh8bwF+GCK1KY9pQ5FZNo7+ha5YzCzJcC5wENAu7sPtzHvBtoLnHM9cD3A4sWLjz/CsGUqb5eeWqZEpi13f32e4q8VqLsLuCZn/27gqGkTKkm0vZnI7CaG1u0gftpcLHJ0672ITI2iB6CbWSNwG3CDu3fnHnN3B/JmNuMewDl8gch3556SKRGZpsyMmuctwHsGSW/bV+pwRKa1opIpM4sTJFLfdffbw+LO4fEJ4fPktDUfo5tPY6ZEZDqLLZ6NNdUy9MQOgu+0IlIKxdzNZwRN5+vd/XM5h+4C3hxuvxn40cSHB82fuYLokhbiz8/Ti6h5pkRkGrOIUXP2gmASz87usU8QkUlRTMvUi4A3ApeNWu/qU8AVZrYRuDzcn3A1LzmJuc/dQKSx5qhjXStvpv/bayfjbUVEKkJ8WTtWGye5bkepQxGZtsYcgO7uDwKFRja+bGLDObbY8tmknz5ygc++rz9G/RvPmcowRETKhsWiJM6Yx9Bj28gc7CPa0lDqkESmnbKfAT3X7F+95aiyfPNPiYhMJ/Hl8yEWIbmuouYeFakaFZVMRdsbsZm1R5SlntzDTruRwbs3lCgqEZHSitTGiZ/aTurZPWT7hkodjsi0U1HJFMCcx95Bw99cOLKf7QxWTh+44+lShSQiUnI1Zy0Ed5JP7Sp1KCLTTsUlU7GTWqh79fKjylOrO3RrsIhMW5GmWmJLZpN8pgNPpksdjsi0UnHJFHB4Is8cqdUdDOjOPhGZxmrOXgipDMlndpc6FJFppWqSKYDkwxp8KSLTV3R2E9F5zSSf2pl/PVMRmRSVmUxF84ftvUkABu5YT/Kxjrx1RESqWc3zFuH9SVKbOksdisi0UZHJVKEFPbM9STJ7+9j/mlvZd9V3pjgqEZHSi85vIdrWxNBj2/BUptThiEwLFZlMFerm894kQ/c/B4A1Hz1juohItTMzas5fig8kNSu6yBSprmSqZwgfSAVVZtZNZUQiImUj1j6D2JLZDK3bQbY/WepwRKpeZSZT0QLdfH0pSIWDLgskXCJSHczsFjPbY2brcsr+xcyeNrPHzewOM2spcO4WM3siXGt01dRFPXVqX7AEss7QY1tLHYpI1avIZKrQmCmyPjJGQKvMiFS9bwBXjSq7Dzjb3Z8PbAA+dIzzL3X3Fe6+cpLiK6nIjDoSZ8wntXE3mf19pQ5HpKpVZDJVsNXJXS1TItOEu/8a2D+q7F53H56x8vfAwikPrIzUnLMI4jEGVz1X6lBEqlp1JVPp7OG7V5RMiUx3fw7cU+CYA/ea2aNmdn2hFzCz681slZmt6urqmpQgJ5PVxKk5ZxGZnQdI7zxQ6nBEqlZlJlOF5pkaTB9umSpQR0Sqn5n9A5AGvlugyovd/TzgauCdZvbSfJXc/WZ3X+nuK9va2iYp2smVOGM+1lTL4COb8ayW3BKZDBWZcRQaM+WDaTwZtkypYUpkWjKztwCvAN7gBRbsdPed4fMe4A7ggikLcIpZNELtC5aQPdCviTxFJklFJlOFuvCye/vp+cdfBzv6AiYy7ZjZVcAHgFe6e3+BOg1m1jS8DVwJrMtXt1rElswOJvJcvVUTeYpMgqpKpsg4pMNuPq1LJVLVzOx7wO+A081sh5m9DbgJaALuC6c9+HJYd76Z3R2e2g48aGZrgYeBn7r7z0rwEabMERN5PqmJPEUmWqzUAZyQAvNMHSGjpimRaubur89T/LUCdXcB14Tbm4FzJjG0sjQykecTO4ifNo9IfaLUIYlUjTFbpgpMjHejme0Mv/mtMbNrJjfMUTEVcaeep9UyJSKSSxN5ikyOYrr5vsHRE+MBfD6c8G6Fu9+d5/jkKWZGTnXziYgcITKjjsTyeZrIU2SCjZlM5ZsYr9R8KH1UWcM7zz+yQN18IiJHqVmxGKuJM/DgBjyrL50iE2E8A9DfFa5/dYuZzSxUaTImvYuePJOmj11C++b3jJQlLjxyomNXy5SIyFGsJk7txaeQ3ddL8gkNRheZCCeaTH0JOAVYAXQAny1UcTImvTMzZnzkEmInH87hrG7UWHq1TImI5BVf0hYMRl+zjcwBdfeJjNcJJVPu3unuGXfPAl+hDCa8s7r4kQVqmRIRKaj24lOwRIyB36i7T2S8TiiZMrN5ObuvpgwmvLMZNUfsu1qmREQKitQmqL14mbr7RCbAmPNMhRPjXQLMNrMdwEeBS8xsBcE841uAv5zEGIsSmdNwZIFapkREjim+ZDapsLsvtmgW0daGsU8SkaOMmUwdz8R4pRQdnUxpnikRkTHVXnwKfbsPMfDgBhpecQ4WqcyFMURKqWr+11hL7RH76uYTERnbEd19j6u7T+REVHwyNesXb2Lmra/FRk3kmd3Vw74/+QGe1qKeIiLHEl8ym9jJsxlau02TeYqcgIpPpmpftpT6152d99jgD58i/eyBKY5IRKTy1F40fHffM7q7T+Q4VXwylWvOE391VFm2q4/BezaWIBoRkcox0t23v0/dfSLHqaqSqfjZ7cTOOnJi0L2XfIN913xX3X0iImM4ortvb2+pwxGpGFWVTAG0r3snjX978eGCcCB67xceKlFEIiKVo/aiZVhtnP4HniI7mCp1OCIVoeqSKQBiR3+s7vffi6fUOiVSLcJ1QfeY2bqcslYzu8/MNobPedcNNbM3h3U2mtmbpy7q8hepjVN/2Zl4f5KBXz2NZ3VntMhYqjOZilre4syuHrKHBjnw1jtJbzs4xUGJyAT7BnDVqLIPAve7+6nA/eH+EcyslWDy4QsJlsL66LEWa5+Oom1N1F68jEzHQYYe3VLqcETKXlUmUxbN/7EyO7o59J576P/GGgZ/9MwURyUiE8ndfw3sH1V8LfDNcPubwKvynPpy4D533+/uB4D7ODopm/YSp80lvnweyXU7SD3XVepwRMpaVSZTBVumdnSTerwTgMjM2rx1RKSitbt7R7i9G2jPU2cBsD1nf0dYdhQzu97MVpnZqq6u6ZdQ1F6wlOicGQw8uEHzT4kcQ5UmU/k/VnZXD4T9/57SPCoi1czdnWD90PG8xs3uvtLdV7a1tY19QpWxaIS6S8/A4jH6f/kUPqQB6SL5VGUyZQVaprI9Q4eXmdHafSLVqNPM5gGEz3vy1NkJLMrZXxiWSR6R+gR1l52B9w0x8D/PaEC6SB5VmUwVapny3iRkgiRKd/aJVKW7gOG7894M/ChPnZ8DV5rZzHDg+ZVhmRQQmzOD2gtPIb3zAEOPbS11OCJlpzqTqTxNLQ9cAAAdQ0lEQVRTI8BwMqWWKZFqYGbfA34HnG5mO8zsbcCngCvMbCNwebiPma00s68CuPt+4BPAI+Hj42GZHEP89LnET5tL8vHtpLbuLXU4ImUlVuoAJkOhbr7kb7eT3rAPAFcyJVLR3P31BQ69LE/dVcDbc/ZvAW6ZpNCqkplRe9EpZA/0MfDrDUT+sJZoa2OpwxIpC9XZMlWgSz+1tjNnR8mUiMjxGBmQXhOj/+fryBzqL3VIImWhKpOpzK6esSupZUpE5LhFGmqof/nZAPT/fB3Z3sESRyRSelWZTKW3jD27ubr5REROTLS5nvorz8ZT6SChGkiWOiSRkqrKZKrmxYuPKotftPDIAt3NJyJywqKzGqm//Gyy/UP0/3yd5qCSaa0qk6mGd1/A3D1/x5x1fw2ANcSJLWs9oo5apkRExifWPoP6l51J9lA//fc9iafSpQ5JpCTGTKbGszJ7qVgkQrStAWtMAMFs55Fwe4QGoIuIjFts/kzqLllOZm8P/fc/pS+qMi0V0zL1DU5gZfZyMJxMkcwQmXvkLbz6Dy8iMjHiJ82m9sWnkek4xMCv1uNZXV9lehkzmRrHyuwll9saFT2p+ciDSqZERCZMYlk7tRedQnr7fgZ+s0HLzsi0cqJjpopZmR0o8arriSgANZcvJTqqZSq5ahfJR7Qcl4jIREmcMZ+aFywhvbkraKHSl1aZJsY9AH2sldlLueq6mdH+7N/Qeud1RBfMOOJY6uGddF3wlSmNR0Sk2tU8fxE1FywlvXUf/fetw4c0KF2q34kmU8WszF4WYktbiTQkiD+vnfq3rjjqeP931jJw19MliExEpDrVnLWAupeeTmZPN333PE62f6jUIYlMqhNNpopZmb3stNz8RzS883yoPbwk4YE33sH+a79PaoMW7hQRmSjxU+ZQf/lZZHsG6fvpWi09I1WtmKkRil6ZvdxZLErLTX9IbOnRMznsOf0mANyd7H79pxcRGa/Ygpk0XP08SGfp/+laMl1FLPUlUoGKuZvv9e4+z93j7r7Q3b/m7vvc/WXufqq7X+7uo+/2K2+xwh97/2tupWPWZ3DNkC4iMm7R2U3UX3MOxGP0/exxUjsq68+FSDGqcgb0sViBZMqH0gzeGYyf8gEtjSAiMhGizXU0/OE5RGbUMfCLp0hu6ix1SCITalomU4XuLsldINmH1DIlUonM7HQzW5Pz6DazG0bVucTMDuXU+Uip4p0uIvUJGq5+PtG5Mxj8zQYGH3lOc1FJ1YiNXaX6+ED+ZCrbkdOfn1QyJVKJ3P0ZYAWAmUWBncAdear+xt1fMZWxTXeWiFF/xdkMPvQsyXU7yOztoe4PlhOpT4x9skgZm54tUwW68DI7DydTrmRKpBq8DHjW3beWOhAJWDRC3QtPpfYlp5Hp6qHvrsdIdx4qdVgi4zI9k6nu/HOeHPiz2w/XUTIlUg2uA75X4NjFZrbWzO4xs7PyVSjpCg5VLrGsnYZXrMBiEfrveYKhJ3cSzAEtUnmmZzLVV8Tgcs3aK1LRzCwBvBL47zyHVwMnufs5wBeBO/O9RilXcJgOoq0NNPzRucQWtTL08GYGfvU0ntK1VyrPtEymiqGWKZGKdzWw2t2PunXM3bvdvTfcvhuIm9nsqQ5QwGpi1F12BjUrl5Deupe+H68hc7Cv1GGJHJdpmUy13vY6mj//8pH9hr8+/6g6SqZEKt7rKdDFZ2ZzzczC7QsIroX7pjA2yWFm1DxvEfUvfx4+lKbvx2tIrt+lbj+pGNPybr6615wJQGrNbvq/uZb4yvlHV1IyJVKxzKwBuAL4y5yydwC4+5eB1wJ/ZWZpYAC4zvWXu+Ri81pouPZcBh7cyODvnyW1ZS91Lz6NSFNtqUMTOaZpmUyNCNfo88Gj++jVMiVSudy9D5g1quzLOds3ATdNdVwytkh9DfVXnEVqYyeDD2+m985HqV15MvHl8wgbE0XKzrTs5hsWW9YKQKS1jsjcxiOO9d38KOltB/OdJiIik8jMSJw2l8ZXnUd0zgwGf/8s/T97gmzPYKlDE8lrWidTje+9mNYfvo66151F/Rued8SxwdvWs+/l3yF7YED99iIiJRBprKX+yrOpfeEyMvt66b3zUZJPayyVlJ9pnUxZNELdH5+JmTHj01dQe+3pRxxPP72XjtZPs++q72jhYxGREjAzEqfPO9xK9buglUp3/Ek5mdbJVC6LRmi97U9p+/3bjzo2dO+zHHrPPcH2b7aSfHjHVIcnIjKtHdFKtb+PvjtXM/jQs/iQFqWX0lMylcOiEaILZ+Q9NnDH02R2drP3pV9n75XfnuLIRERkpJXqj19A/LS5JJ/aRe9tq0g+3aFFk6WklEyNVhPNW5zd3UvqiWDuPz+UfzkaERGZfJHaBHUvPJWGV55LpKWewd9tou/Hj5HerTX+pDSUTI1iifzJFEB2eE2/mH5sIiKlFp3VSP3Vz6fukuX4UJr+ex6n/4H1ZHt1159Mrek9z1Qex0qmDvzpD8esIyIiU8fMiJ/cFqzv98QOkk/soHf7PhKnzyPxvEVE6hOlDlGmASVTo8WLSJTiapkSESknFotSe+5JJE5tZ2jNNpLrd5F8ZjeJ5fNIPG8hkTolVTJ5xpVMmdkWoAfIAGl3XzkRQZWSRcdOlEzdfCIiZSnSWEvdi0+j5vmLgqTqqZ0kn+kgccZ8EmcvJFIbL3WIUoUmIiu41N1XVEMiNax9yw3M6/3wyP7o6RJ8SHNOiYiUs8iMOupeejoNr34BscWzgu6//36EwUe3aDoFmXDq5ssjdlLLkftntR2x7wMpMp29RNuPXIJGRETKS7S5nvo/WE7mnLCl6vHtJJ/aReK0dhJnLtAiyjIhxtsy5cC9ZvaomV2fr4KZXW9mq8xsVVdX1zjfbmo1ffxSYmfMJtJYc+SBjLN77r+SfKyDbH+yNMGJiEjRoi0N1F9yBg3Xnkf8pFkk13fQe9sj9D+wnvSe7lKHJxXOxrPGkZktcPedZjYHuA94t7v/ulD9lStX+qpVq074/Uop+chOui74yhFliRcuIvnb7TS89yKa//VKLBLB3bWyuUgOM3u0GoYBVPL1S46W7RsaGaROMk20rYnE2QuILZ6NRXQNl0Cx169xtUy5+87weQ9wB3DBeF6vnCXOX8C8A39P0ycuHSlL/nY7AH2f/z3J3+0g9UQnHc2fZOBHT5cqTBERKUKkoYbalSfT9LoLqL3wFLKDKQYeeJre21Yx9OROsoMaVyXFO+ExU2bWAETcvSfcvhL4+IRFVoYiLXVE5zflPTZ037Nku/rxniTJh3ZQd+3yKY5ORIaNdaexBc3HXwCuAfqBt7j76qmOU0rP4lESZ84nvnwe6e37SD65k6GHNzO06jliS2aTOG0u0bnN6nGQYxrPAPR24I7wFywG/Je7/2xCoipjVpf/ttrkI7vIbNdSBiJl5FJ331vg2NXAqeHjQuBL4bNMUxYx4ifNJn7SbDL7+0ht2E3y2T2kN3cRaaolftpc4qe2a74qyeuEkyl33wycM4GxVIYCfelDd28c2fYeDUoXKXPXAt/yYNDo782sxczmuXtHqQOT0ou2NhC96BRqVi4htWUvqQ27GXp0C0OrtxJb3Er81LnEFrRgEc05KAFNjXCconMaxqzjvUqmREps+E5jB/7T3W8edXwBsD1nf0dYpmRKRlgsSmJZO4ll7WQO9pPasJvUpk7SW/dhNTFiJ7cRX9pGdM4MdQNOc0qmjlPikiXM3f5erCFBR+unAZjxyZfR/aH7R+pke4Z0V59Iab04905jM3v6WHcaFxJO+XI9wOLFiyc6Rqkg0ZZ6ohcspeYFS0jvOEBq8x5SGztJPd2BNdYQXzonSKxmjv2FW6qP2iiPk5kRXdhMZGYdAImXLKb2NWccUWfwtvXsin+c7P5+Mrt7ShGmyLRWxJ3GO4FFOfsLw7LRr3Ozu69095VtbW2jD8s0ZNEI8ZNmUX/pGTS9/kJqX3IakeZ6kk9sp+/O1fTe+ShDa7eROdjHeKYeksqilqlxmNf9ISwRxWpiNP/HH3Lor396+GDG2X3S/8N7k9S//TxmfuWVZHZ2M3DHehreeYFarUQmSZF3Gt8FvMvMvk8w8PyQxkvJ8bJ4bKQbMDuQJL1lL6nNXQyt3srQ6q1EZtQRWzyL2EmziLY16bpfxZRMjUOk6fDM6I1/dT4Nbz+P/a+5lcGfbAAOj53q/+pqmv7hJXSe+kVIZ6m5fCnx5fqWKzJJ8t5pbGbvAHD3LwN3E0yLsIlgaoS3lihWqRKRukSwmPIZ88n2DZHevo/U1mCqheS6HVhdIhi8ftLsYKqFqDqGqomSqQlk8ShE83/zOPT+eyGdBSC7b2AqwxKZVgrdaRwmUcPbDrxzKuOS6SPSUENi+XwSy+fjQ6lgjNXWvaSe3UPqmd0QjxKb10JswUxiC2cSadT6gJVOydQEy3T25S0fvG39yHb2gJIpEZHpwGrixE+ZQ/yUOXg6Q3rXQdI79pPecYD0tn0ARJrriC1sJbZgJtH2ZiymVqtKo2RqgrXcdA3J326n5xP/Q7arHwCbWYsfGBypk3psNzUvOYlIs76NiIhMFxaLEl88i/jiWbg72UMDpHceIL1jf7BO4JM7IRohOreZ2LxmYvNaiLQ2aq3ACqBkaoIlXjCfxAvmU3v1MjpP/SKzfvEmkg9uo+fGX43U6fnIAwze9Qy1154OqSwNf30+0fZGfCiNpzJEGmsKv4GIiFQ8MwumW2ipp+asBXgqQ3r3IdI795PZdZChVQcYAkhEibU3E53XQmxeM5GZDRrIXoaUTE2S2LJZLPAbAai57GTiK+Yy+KOn6f/6GgBSq3aRWrULgL4vPUL75vfQ0fRJEhcvpO23by9V2CIiUgIWjxJf1Ep8USsA2f4kmd0HSXccIt1xkPT2/QwBVhMj2t5MtH0G0TkziM5q1GD2MqBkagqYGXXXLqfu2uUjyVSubFc/HU2fBCD5ux1THZ6IiJSZSH2CyNI5xJfOASDbO0R690EyHQdJd3aPjLciGiE6uzFIsObMIDanCavJv4asTB4lU1MsenILmecOHrNOZncP0blNUxSRiIiUu0hjDYll7bCsHQhbrvZ0k+k8RHpPN8kndkA4SWikuY5oWxPR2U1E25qCrkG1Xk0qJVNTbM6q67G6OPte+wOG7t5I+4Z303naF4+os3veZ5n5zVcx+IvNxM+aQ+PfXAi1MZL/u434mW1EWutLFL2IiJSDSH2CyJLZxJfMBsBTGTJ7e8h0dpPZ2xNMx7BpT1A5akRbGw8nV7Maicyo08D2CaRkaooNJ0KzfnQd2c4+ogtm0L7tvfR96RGSv91O8n+2AnDgzXcCMACknurCokb/19fQ+LcX0/zZl5cqfBERKUM2PHfVvBYA3B3vGyLT1RMkWV09JDfuhvXBWF1iEaKtjURaG4jOaiQ6q5FIS71asE6QkqkSsViU6IIZAMQWNdP8z5eTXNNB17n/eVTdgW+tHdlOPdFJprOXaHvjlMUqIiKVxcywxloijbXETw5W3PCskz3YT2ZfL9l9vWT295LatIfU0+FKShEj0lJ/OMmaWU9kZgORukQJP0llUDJVRuLnzKX1ttdhDQn2XfUdAJpvuoZD77p7pM7QfZvZPfdfaf63qzn0gftoePt5NH/mClJP76X/22uZ8bFLj1jmRkREBMAiRrS1gWhrA5wajL1yd7LdgyPJVWZfL+md+/FNnYfPq40TmRkmV60NRFsaiLTUYXGlEMNsKle1Xrlypa9atWrK3q+SZQ8OBN8SZtSSerqLoXufZfBHzzD0y+eOed7M77yGuuvOVlOtlA0ze9TdV5Y6jvHS9Uumk+xgkuz+fjIH+sge6Auf+yGTHaljDTXBYPeWeiLhI9pSX1V3ExZ7/VJaWaYiLXUj2/HlbcFj5XysIc7g3RshEyTB0YUzyOzoHql74M9up+cz/0vslJkM3vUMMz5xGQ3vWEnfLY/hvUlmfPSSqf4oIiJSYSK1CSLzE8Tmt4yUedbJ9gySPdhP9lB/0GV4sJ/kht0ja89C2JI1o45Ic/iYUUekuZ5IU23VftFXMlVBal64mJq7/j8Akr/fTuz57VjE6P/+OtJPddH/9TVk9/aTfryT9ONBE233h++n+8P3j7yG9wzR8J6LSD+9F+9LUnPpyUSaa0k+spPokhaibQ0l+WwiIlLeLGJEm+uINtcBs0bK3R3vHSJzMEiwst0DZA/1k96xH9+YynkBgnFcM+qINI16bqyt6DUJlUxVqMRFi0a2G95yLgB1f3IW6Y37GLx7I6m1nTS87VxS6/fSf/OjI3V7P/s7ej/7u5wXikIyA0DsrDYa330htdeeTvKhnUSaa0i8aDE+mCb50A5qLlmCxaJT8wFFRKQimBnWVEukqRbCGdyHeTJN9tAA2e4BMof6g+2eQVJ7uiGVOfJ1GmqC5Gr40Vg78rpWGy/rZXTGNWbKzK4CvgBEga+6+6eOVV9jDkrH0xm6P3Q/6Q37SG/cR3r9XmpftZzUuj1kNu0vfGJNFIaCX/jIgiZqL18K8SjeM0Ts9NnEn9/O0P9uIzKjhoZ3rCTSGnRP+kAaa0pgkQjZviRWF8MilfutQ06cxkyJyGjujg+lyXYP4D2DQWvW8HPvID6QOvKEWIRIeHeiNdYQaawh0jC8XYvVTU6yNeljpswsCvw7cAWwA3jEzO5y96dO9DVl8lgsSvO/XJn3mGeyZPcP0P+VR8keGiL97H68L0XiooUkf7udzLZDpJ87QHZnD/3fXJv3NQB6PvY/R75nUwKiEfzgIJEFTcROm0V0TgPZ3iRkneiiZhhKk903QPy8eVhDHGtM4ANpcCcyqx6rjZE9MED8zDYwg6zjWcdiEaw2hmeyWCJKZLh7MufLgcUieDJDZFY9xCLBMQeyfng7YtiMmrL+xiMiUm3MLBhbVRuHOTOOOu6pDNnewSCx6hkk2zsUJFu9g2T3dEMyfeQJESPSUBMkVw01QStXznOkoQZLTF5n3Hhe+QJgk7tvBjCz7wPXAkqmKoxFI0TbGmj68EsL1gmaaofAncxzB4kubsbTWVKrO7D6OJmtB0k93gkO2YODxE5qJrt/gExnH5aIknq8M0jK1u8N5zKpJfn7HZDKQiLK4E82TOEnHiViYBxOtIbLohYkW9HI4X2zwwlb7rHcmYRz87LhJC33sNnh/eHt3Hr5ErtCuV6RdQsmi/mKx3jN2PLZzLr9ugIBlZ6ZLQK+BbQT/Kve7O5fGFXnEuBHwPDtsbe7+8enMk4RKcziUaIzG4jOzD+O15PpIMHqG8R7h4Lt3kG8b4j0roP4QDL4358rHiVSn6DusjOJtkzsSiLjSaYWANtz9ncAF46uZGbXA9cDLF68eBxvJ6VkiRjRtuDXJTrn8IShsUXN435tdye7pw9iEbxnKGiurY2R3T+A96eCBK6zL1j6IHx4MoMPpDAzst1D+GD66BdOZiAWIbuvP9iP2EjyMpzQeMbxAwO4c/j1ATJZPOvBXZNZxzPZINHK+uFkI5tzbPg/bW63eaGy4f1w23Pr5et1L9QVX2zdQj35xdYdVS+6pCVPpbKSBt7n7qvNrAl41Mzuy9Nq/ht3f0UJ4hORcbJEjGhrLJgzKw/PZvH+JNm+IbxviGxfMnwempQWqkkfgO7uNwM3QzDmYLLfTyqPmR2e0X3W4W8Lkebake342VMdlVQqd+8AOsLtHjNbT/DlT63mItOERSIjM8BPhfGMCN4JLMrZXxiWiYiUBTNbApwLPJTn8MVmttbM7jGzswqcf72ZrTKzVV1dXZMYqYhUsvEkU48Ap5rZyWaWAK4D7pqYsERExsfMGoHbgBvcvXvU4dXASe5+DvBF4M58r+HuN7v7Sndf2dbWNrkBi0jFOuFkyt3TwLuAnwPrgR+4+5MTFZiIyIkyszhBIvVdd7999HF373b33nD7biBuZrOnOEwRqRLjGjMVXoTuHrOiiMgUseDWxa8B6939cwXqzAU63d3N7AKCL5b7pjBMEakimgFdRKrNi4A3Ak+Y2Zqw7MPAYgB3/zLwWuCvzCwNDADX+VSu+i4iVUXJlIhUFXd/kMIzcw3XuQm4aWoiEpFqp/U9RERERMZByZSIiIjIOIxroePjfjOzLmBrkdVnA3snMZypUOmfQfGXVrXEf5K7V/y8Asd5/YLy//cr5/jKOTZQfONVzvGNjq2o69eUJlPHw8xWVfpK85X+GRR/aSn+ylbun7+c4yvn2EDxjVc5x3eisambT0RERGQclEyJiIiIjEM5J1M3lzqACVDpn0Hxl5bir2zl/vnLOb5yjg0U33iVc3wnFFvZjpkSERERqQTl3DIlIiIiUvaUTImIiIiMQ1kmU2Z2lZk9Y2abzOyDpY4nHzO7xcz2mNm6nLJWM7vPzDaGzzPDcjOzfws/z+Nmdl7pIh+JdZGZPWBmT5nZk2b2nrC8Ij6DmdWa2cNmtjaM/2Nh+clm9lAY561mlgjLa8L9TeHxJaWMf5iZRc3sMTP7SbhfafFvMbMnzGyNma0Kyyrid2iylPv1K9+/WYnjKfpaWkbx3WhmO8Of4Rozu6ZEsR3XdbyM4iuXn99x/R05JncvqwcQBZ4FlgIJYC1wZqnjyhPnS4HzgHU5ZZ8BPhhufxD4dLh9DXAPwXphFwEPlUH884Dzwu0mYANwZqV8hjCOxnA7DjwUxvUDgkVrAb4M/FW4/dfAl8Pt64BbS/1vEMbyt8B/AT8J9yst/i3A7FFlFfE7NEk/j7K/fuX7NytxPEVfS8sovhuB95fBz+64ruNlFF+5/PyO6+/IsR7l2DJ1AbDJ3Te7exL4PnBtiWM6irv/Gtg/qvha4Jvh9jeBV+WUf8sDvwdazGze1ESan7t3uPvqcLsHWA8soEI+QxhHb7gbDx8OXAb8MCwfHf/w5/oh8DIzO+ZiuJPNzBYCfwh8Ndw3Kij+Y6iI36FJUhHXr3JynNfSKVcgvrJwAtfxcomvLJzA35GCyjGZWgBsz9nfQRn98MfQ7u4d4fZuoD3cLuvPFHYZnUuQlVfMZwi7yNYAe4D7CFoEDrp7OqySG+NI/OHxQ8CsqY34KP8P+ACQDfdnUVnxQ3DhudfMHjWz68OyivkdmgSV8Bnz/ZuVm0K/Q+XkXWF39S2l7IYcVuR1vGRGxQdl8vM7zr8jBZVjMlUVPGgfLPt5J8ysEbgNuMHdu3OPlftncPeMu68AFhK0CCwvcUhFM7NXAHvc/dFSxzJOL3b384CrgXea2UtzD5b779A0dcx/s3JTpr9DXwJOAVYAHcBnSxlMuV/H88RXNj+/ifo7Uo7J1E5gUc7+wrCsEnQOd1uEz3vC8rL8TGYWJ/gF/6673x4WV9RnAHD3g8ADwMUEXUex8FBujCPxh8ebgX1THGquFwGvNLMtBF1BlwFfoHLiB8Ddd4bPe4A7CC5GFfc7NIHK/jMW+DcrN4V+h8qCu3eGf4SzwFco4c/wOK/jZRFfOf38hhX5d6SgckymHgFODUfTJwgG295V4piKdRfw5nD7zcCPcsrfFN7NdBFwKKcJtiTC8TZfA9a7++dyDlXEZzCzNjNrCbfrgCsI+uMfAF4bVhsd//Dnei3wy/AbW0m4+4fcfaG7LyH4Hf+lu7+BCokfwMwazKxpeBu4ElhHhfwOTZKyvn4d49+s3BT6HSoLo8b6vZoS/QxP4Do+pQrFV0Y/v+P9O1LYRI2Kn8gHwV0/Gwj6Lv+h1PEUiPF7BM2TKYI+1bcRjGG5H9gI/AJo9cN3DPx7+HmeAFaWQfwvJmj6fRxYEz6uqZTPADwfeCyMfx3wkbB8KfAwsAn4b6AmLK8N9zeFx5eW+t8g57NcwuG7+Som/jDWteHjyeH/q5XyOzSJP5eyvX4V+jcrcUxFX0vLKL5vh7/DjxMkLvNKFNtxXcfLKL5y+fkd19+RYz20nIyIiIjIOJRjN5+IiIhIxVAyJSIiIjIOSqZERERExkHJlIiIiMg4KJkSERERGQclUyIiIiLjoGRKREREZBz+fyn6Y5y6kI7bAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "# 定义网络结构\n",
    "model = Linear(2)\n",
    "# 定义优化器\n",
    "opt = SimpleBatchGD(init_lr=0.01, model=model)\n",
    "train_and_plot(opt, 'opti-loss.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，loss在不断减小，模型逐渐收敛。\n",
    "\n",
    "**提醒**  \n",
    "在本小节中，我们定义了两个实验：2D可视化实验和简单拟合实验。这两个实验会在本节介绍的所有优化算法中反复使用，以便进行对比。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**与Paddle API对比，验证正确性**\n",
    "\n",
    "分别实例化自定义SimpleBatchGD优化器和调用paddle.optimizer.SGD API, 验证自定义优化器的正确性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model1 parameter W:  [[-4.080414 ]\n",
      " [-1.3719953]]\n",
      "model2 parameter W:  [[-4.080414 ]\n",
      " [-1.3719953]]\n",
      "model1 parameter W after train step:  [[-3.196255 ]\n",
      " [-1.0747064]]\n",
      "model2 parameter W after train step:  [[-3.196255 ]\n",
      " [-1.0747064]]\n"
     ]
    }
   ],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "x = data[0, :-1].unsqueeze(0)\n",
    "y = data[0, -1].unsqueeze(0)\n",
    "\n",
    "model1 = Linear(2)\n",
    "print('model1 parameter W: ', model1.params['W'].numpy())\n",
    "opt1 = SimpleBatchGD(init_lr=0.01, model=model1)\n",
    "output1 = model1(x)\n",
    "\n",
    "model2 = nn.Linear(2, 1, paddle.nn.initializer.Assign(model1.params['W']))\n",
    "print('model2 parameter W: ', model2.state_dict()['weight'].numpy())\n",
    "output2 = model2(x)\n",
    "\n",
    "model1.backward(y)\n",
    "opt1.step()\n",
    "print('model1 parameter W after train step: ', model1.params['W'].numpy())\n",
    "\n",
    "opt2 = optimizer.SGD(learning_rate=0.01, parameters=model2.parameters())\n",
    "loss = paddle.nn.functional.mse_loss(output2, y) / 2\n",
    "loss.backward()\n",
    "opt2.step()\n",
    "opt2.clear_grad()\n",
    "print('model2 parameter W after train step: ', model2.state_dict()['weight'].numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，在一次梯度更新后，两个模型的参数值保持一致，证明优化器实现正确。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 7.3.2 学习率调整\n",
    "\n",
    "学习率是神经网络优化时的重要超参数。在梯度下降法中，学习率$\\alpha$的取值非常关键，如果取值过大就不会收敛，如果过小则收敛速度太慢。\n",
    "\n",
    "常用的学习率调整方法包括如下几种方法：\n",
    "* 学习率衰减：如分段常数衰减（Piecewise Constant Decay）、余弦衰减（Cosine Decay）等；\n",
    "* 学习率预热：如逐渐预热(Gradual Warmup) 等；\n",
    "* 周期性学习率调整：如循环学习率等；\n",
    "* 自适应调整学习率的方法：如AdaGrad、RMSprop、AdaDelta等。自适应学习率方法可以针对每个参数设置不同的学习率。\n",
    "\n",
    "**动手练习7.2**  \n",
    "尝试《神经网络与深度学习》中第7.2.3.1节中定义的不同学习率衰减方法，重复上面的实验并分析实验结果。\n",
    "\n",
    "下面我们来详细介绍AdaGrad和RMSprop算法。\n",
    "\n",
    "#### 7.3.2.1 AdaGrad算法\n",
    "\n",
    "AdaGrad算法（Adaptive Gradient Algorithm，自适应梯度算法)是借鉴 $\\ell_2$ 正则化的思想，每次迭代时自适应地调整每个参数的学习率。在第$t$次迭代时，先计算每个参数梯度平方的累计值。\n",
    "$$\n",
    "G_t = \\sum^t_{\\tau=1} \\mathbf g_{\\tau} \\odot \\mathbf g_{\\tau},\n",
    "$$\n",
    "其中$\\odot$为按元素乘积，$\\mathbf g_{\\tau} \\in \\mathbb R^{\\mid \\theta \\mid}$是第$\\tau$次迭代时的梯度。\n",
    "$$\n",
    "\\Delta \\theta_t = - \\frac{\\alpha}{\\sqrt{G_t + \\epsilon}} \\odot \\mathbf g_{t},\n",
    "$$\n",
    "其中$\\alpha$是初始的学习率，$\\epsilon$是为了保持数值稳定性而设置的非常小的常数，一般取值$e^{−7}$到$e^{−10}$。此外，这里的开平方、除、加运算都是按元素进行的操作。\n",
    "\n",
    "**构建优化器**  定义Adagrad类，继承Optimizer类。定义step函数调用adagrad进行参数更新。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nndl.op import Optimizer\n",
    "\n",
    "class Adagrad(Optimizer):\n",
    "    def __init__(self, init_lr, model, epsilon):\n",
    "        \"\"\"\n",
    "        Adagrad 优化器初始化\n",
    "        输入：\n",
    "            - init_lr： 初始学习率\n",
    "            - model：模型，model.params存储模型参数值\n",
    "            - epsilon：保持数值稳定性而设置的非常小的常数\n",
    "        \"\"\"\n",
    "        super(Adagrad, self).__init__(init_lr=init_lr, model=model)\n",
    "        self.G = {}\n",
    "        for key in self.model.params.keys():\n",
    "            self.G[key] = 0\n",
    "        self.epsilon = epsilon\n",
    "\n",
    "    def adagrad(self, x, gradient_x, G, init_lr):\n",
    "        \"\"\"\n",
    "        adagrad算法更新参数，G为参数梯度平方的累计值。\n",
    "        \"\"\"\n",
    "        G += gradient_x ** 2\n",
    "        x -= init_lr / paddle.sqrt(G + self.epsilon) * gradient_x\n",
    "        return x, G\n",
    "\n",
    "    def step(self):\n",
    "        \"\"\"\n",
    "        参数更新\n",
    "        \"\"\"\n",
    "        for key in self.model.params.keys():\n",
    "            self.model.params[key], self.G[key] = self.adagrad(self.model.params[key], \n",
    "                                                               self.model.grads[key], \n",
    "                                                               self.G[key], \n",
    "                                                               self.init_lr)                "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**2D可视化实验**  使用被优化函数展示Adagrad算法的参数更新轨迹。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x1 initiate: [3.], x2 initiate: [4.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAF5CAYAAACFu8BrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XmcZHdd7//X9yx1aq/qql5mnwlZwbAEEFEUWQS5qIj7/kMEA8pPBMQFvcIVF9DL5aJX+UkQxQVBrhB20QAJYU1I2JKQkEy22ae7q2uvOvv5/XFOnanu6Z6lMzPV3fN58vjyXc6p6m9NZvq861vnnFJRFCGEEEIIsRlpk56AEEIIIcR6SZARQgghxKYlQUYIIYQQm5YEGSGEEEJsWhJkhBBCCLFpSZARQgghxKZlTHoCAEqph4AuEAB+FEVPnuyMhBBCCHE+rXXsV0r9BvCKZPzjURT9zqmeZ0MEmcQzoyhanPQkhBBCCHHBLDv2K6WeCfwo8Pgoihyl1OzpnkA+WhJCCCHERvFrwJujKHIAoiiaP90DNkqQiYD/UkrdrpS6dtKTEUIIIcR5t9qx/wrg+5RStyilPquU+s7TPclG+Wjpe6MoOpwsId2glLoniqKbx3dIXuS1AIVC4UlXXXXVJOYphBBCXHC33377YhRFMxfq5ymlzsX3F90F2GP966Ioum6sf9KxnziX1ICnAt8JvF8p9ajoFN+ntCGCTBRFh5N6Xil1PfAU4OYV+1wHXAfw5Cc/Obrtttsu+DyFEEKISVBKPTzpOayDfaqLd9Y49h8CPpgEl1uVUiEwDSys9TwT/2hJKVVQSpVGbeC5wJ2TnZUQQgghzpdTHPs/BDwzGb8CyACnvBBoI6zIzAHXK6Ugns+/RlH0yclOafMJw5AgCAiCAN/3V22vVkaPW60+kxJF0Un90dh4vVYBTmqP12cj+Tu0rF7ZXq1omrasHrVHZWX/VEXX9TXrtYphGKu2NW3i7zOEEOJ8WfXYr5TKAH+vlLoTcIEXnepjJQC1ngPGpJ2jz+6EEEKIDWWtY7JS6vYLeY+1c3ScvSBz3ggrMgJ4+tOfjmEYGIaBaZroup7Wo/FRGX/nvrJe653+qVYGxlcOVltdWLk6MRpfa0VjtRWOUxU4efVkvdZa5TldWbm6tHJ81A6C4KRVqLVWscZXuNZaCTvVytlobLV6vARBgOd5J43ffPPNp/qjEkKILWHLBpmnP/3pWJaVlkwms6w96o/ao75pmqv2R7VhGOnjTNNctn3U1nV90i9fiHUJggDXdfE8L61HxXVdHMfB931c103LaNt433GcZWPjfcdx0v6o7TiOBC8hxLpsyiBzzTXX8PnPf37S0xBiy9F1nVwuRy6Xm/RUThKGYRp6bNtO65XtU5XhcHhSezgc8qlPfWrSL08IsU6bMsgIIS4+mqZNJGQFQcBwOGQwGKTBZzAYLKv7/T62bTMYDOj3++m2leWGG264oHMX4mIgQUYIIU5B13WKxSLFYvG8/Qzf99MQNKpHZWW/3+/T6/WW9TudDjfddNN5m58QG5kEGSGEmDDDMCiXy5TL5XP6vK7r0uv16PV6dLtd+v0+3W437Y/XvV6PTqeTjt14443ndC5CnC8SZIQQYovKZDLUajVqtdojfq4wDNMg1Ol0ltXtdptut7us3el0aLfbfPrTnz4Hr0SItUmQEUIIcVqaplEqlSiVSuzYsWPdzxMEAZ1OJw06o3ar1UrHRuOjtoQhcSoSZIQQQlwwuq4zNTXF1NTUup/Dtu005LRarWV1s9lM+6MiH5NtbRJkhBBCbCrZbJZsNsvc3NxZPzYMw/QjsKWlpTTsNJvNtIz6S0tLfPaznz0Pr0CcSxJkhBBCXDQ0TaNSqVCpVNizZ89ZPdZ13TTsLC0t0Ww2aTQay/pLS0tcf/31655fGIbyPWtnSYKMEEIIcQYymQxzc3NnvRI0GAxYWlqi0WiwtLSUlkajkZalpSVuuOEGCTHrIEFGCCGEOI/y+Tz5fJ5du3ZNeipbkkQ/IYQQQmxaEmSEEEIIsWlJkBFCCCHEpiVBRgghhBCblgQZIYQQQmxactXSJhRFEWEYEgQBvu8vK0EQLBsftUfjYRgu64+Pr1WvbIdhmM5hZYmiKN02Xo8XIB0fvZ7x9nh9NpRSJ9Xj7dFljaPxUdE0bVk9aq8sK8d1XT+pvVo9XgzDWDZuGEZaj9rj/fEil2UKIcTJLvogE4YhjuOkxXVdXNdN+57npX3P85btM9q2Wt/zvLQ/qn3fXzY2KqPx8fZ4PV5GY+LiZBgGpmmeFHJGY+P1yrZpmmQymXR81B7Vo/b4+Gp9y7IwTRPLspbtY1nWsm2jIgFMCHE+bcogs7CwwFvf+lYcx8G2bYbDYdq2bTttj8LIyvYomNi2fV5CweiX+coDxGoHFsuDgm5hFIqYhhEX3cQwdAzdwDRMDF3HMHRM3URPxg1dT4qR1pquLevrmha/y9c0NE1HT7brmoauJasEaVtDKS0d00bjoxUIpSX7KHRNR2kKTZ3YplR8x0zF2ArHipUPxWi1Y3kfTl49Wa+1VnkiRitF0bJ+uoI0tpIURqNVJAij8MRYGBGEAVEUEQRhui0IQ4IwONEeX8kKg3QsCAP8pA6CkDDZ5vt+Ou4HAX7gEwYhfuCn/biO24Ef4AUevh+Peb4X75OMe76P5/v4vofvBbjKZzgcnhSWVwbuUTnXDMMgm80uCzejMj4+ao9uPz9q53K5ZeOjMhrP5XJpf2XbNM1z/nqEEBuLWs8S/qQppdJJm6a55i/AbDZL1oOMmSFrWVhmBitjYWUycTsdi/sZM0PGNE/0x8YtM3lXaprJWBJQjLidMU1Mw8DQjUd0IBZikqIowg/iIOR6Xlz8ZFXR83A8FzcJP47n4rgurp+sYiZ9x0tCke+lfcdxTmx347btOLiei21y0puQ8bbneet+Pbquk8/n03Azao/Xo/Hx9vjYqUqhUCCXy8mqkzhn8vn8quNKqdujKHryhZrH+HH2Ebggc96UKzKPvfzR3PjO92OZGXRdn/R0hNgylFKYholpmOSzuUlPB4AgCNLgM3RsbMfGdt2x9mg8rod2PD50HAbOMG0P7WG8PWl3vS7z8/MMh0MGg0Far2dVahRqCoUC+XyeYrGY1qPxQqFwUr9UKi2ri8ViWizLOg9/mkJsPZsyyBi6vmF+yQohzi9d18nruQv2bz4IAgb2kIFtM3SGaXswHNC3h2ndHw4YDIfLxgbDAb3hIKmHLCwsMBgM6Pf7aTlTpmkuCzajUiqVTqpH7XK5fFI9CkmyaiS2qk0ZZIQQ4nzRdZ1SoUipUDznzx2GIQN7SG84oD/oJ/WA3rBPfzigO+jT6/fpDfv0Bn16gwG9QT8eH/To9vscO3aMXq9Hr9ej2+2e0Xl+Sqk08JTL5TTgjNcr2+VymUqlsqydzWblo3Ox4UiQEUKIC0TTNIr5AsV8Aeozj/j5oijCcV06/R69QS+p+6vW3X4vqbt0ej3a7TYHDx6k0+nQ6XTOaLXINM2Tws2oX6lUqFar6dhq7VKpJCtD4pyTICOEEJuUUoqsZZG1LGZr9Uf0XL7v0x306fS6dPo9Ov2k7nVp97pxP2mPSsd1uO+++2i323Q6HXq93mnnOwo3o4AzKpVKhampqWVjtVqNarWajhuGHLLEyeRvhRBCCAzDYKpcYapcWfdz+L5Pp9+j3evQ6nbGQk+HVqdNq9el3e3Q6rbj7f6JINRqtRgMBqd8/nK5zNTU1KqlVqstq0elXq+TyWTW/ZrExidBRgghxDlhGAa1SpVapbqux7ueS7MzCjptljrtOAB1Oyx1WrS6HZqdFkudNs3BgCNHjtBoNGi1Wqc8V6hYLFKr1dKQU6/X0/6o1Ov1dLxer1MqleR8oE1CgowQQogNIWNmmKtPM1efPqvHRVFEd9Cj2W6z1GnR7LRpdlo02i2W2i2WOs2kbrPU73Pw4EGWlpZoNptrfh2KaZrLAs5aZXp6Oq3z+byEnwmQICOEEGJTU0pRLpQoF0rs3bHrjB8XBAGtXodGKw46jXYSeNpNGu0mi6Nxv88999xDo9Gg0WgQhuGqz5fNZtNgMwo3pypTU1Ny8vM5IEFGCCHERUnXdeqVKeqVqTN+TBiGtHtdGq0mjfYSC80lGu0mjVaTxdYSi0nd6HZ58MEHWVxcpNvtrv3z63VmZmbS8r73vU/CzVmSICOEEEKcIU3T0pOiL2PfGT3GdhwW20ssNpfGAs8S80uNpF5kcdjjnnvukRCzDhJkhBBCiPMoa1nsmt3Ortntk57KliTRTwghhBCblgQZIYQQQmxaEmSEEEIIsWlJkBFCCCHEprVhgoxSSldKfU0p9bFJz0UIITaS933yI1z5wqdT+O7LufKFT+d9n/zIpKckxDmx8tivlHq2UuqrSqmvK6U+r5S67HTPsWGCDPCbwN2TnoQQQmwk7/vkR3jFm3+fg8eOEEURB48d4RVv/n0JM2KrWHns//+AX4ii6AnAvwL//XRPsCEuv1ZK7QJ+CPhT4DUTns6mEkURQRDgBwF+4J9UB35Sh2E6Pto/DIO0H4QhwVg/jEKCICRK6rgfEEYRQRgQhmFcoogweWwURkTE/SiKCKMorsNwbDye86g/eg0RUbptNHa2RrcGV0qhFChUOqZpWtpX6kRf0zSUUmgq3pbupyl0TUfTNLRkXNO0eEwpdF1HUxq6rqGSWlMahm6g6/F+uqah6zqGrqNpcT3qG7qBriX7G3F/ND6qTcNI5ycuXm/427cwtO1lY0Pb5g1/+xZ+9nkvmNCshHjk1jj2R0A5aVeAI6d7ng0RZIC3Ab8DlNbaQSl1LXAtwO5tO877hIIgwPFcHNeNa8dJ+7br4HouruelY+547bl4nofre8mYh+d7uF5S/GR7so/n+/jJds/38Xwf13fxk3ZcRvv5+EE85gcBnu+d9z+LC0kptSyQnKnxALSeELSRmYaJoeuYhhEHHSMOOaZhJnU8ljEyaT9jmhjJ9oxhkjFNTNMkY2TImEk/GbcymXifUW1m4rGktsy4mKZJNmOdGLOsuM5kyGYsuZHXeXLo+NGzGhdiA5lWSt021r8uiqLrxvqrHftfCnxCKTUEOsBTT/dDJh5klFI/DMxHUXS7UuoZa+2XvPjrAB79qMujW+/8GkPHZug42LbNwBkydBwcx0nGbeykbbsOjuukY47rLhuPazdpx4HkXAYEXdexzOQAYiQHlNGBxDDjg1DSN9oOOd3A1PMYpo5pGZi6jqHpmFr8bt3U4r6h6xhKx9R19NFYWrR0TFda2o/b8UpDOq60sX68kqCUQlfJNqWlY9poXIvHlVJoKDSVrFyMVjaUGlsBWdGH5LHxqkny9+Cc/XmPpOEmWe0Jo5CIsRWhJPSs7AdRmO4fkqwqRWG8ahWFJ/YLk5WqpIzG/DCI90v290ePG9sWhEE67odB2vfDIC1BGOAFAX6UrLiFAV6yauaFftxP2l7g49kBfuAQTGm0e900+MZhOQnKSXgehfAgCM7Zn7dpmEmoyWAlgScOPhbZjEXOysZj1qgd19mMlY7lrCyWZZFP2tlslpxlkbNy5Kws+WyWrJUln82Rz2bJmJktvWL1yS/ehEIRcXI43zUnN1cT588111zD5z//+Uf0HIVCYTGKoievtu0Ux/5XA8+PougWpdRvA28lDjdrmniQAZ4GvEAp9XwgC5SVUv8SRdEvrvWAex7czzNe+lOnfeLRL9Ls6Jdi8gs2Z2UxOy51w8Qy82RzGSzDxNJNssaJdsYwsPQMlmGQ0U0swySjG8m25W1LNzBHfT3eljEMTC3+CEFceOnKDgoU6BvqlLCNIQhDvNDH9X3cwMcNPJzAwwt8nCAOQU7gxdvG2o7vxfv6Pk7g4vo+TuDh+B627y5vex7O0GFQjmi0m+kbhviNRfymw/XW98ZB07Q04OSsONzkc/mx/omxQjZHLpujkMtRyObJZbMUcwXyuXgsn82n2wr5PMVcASszmaDkei6vf/tb+Kv3/j27Zrez2FrCdp10ey6b5Y9e/toLPi8hzqHVjv0fB66KouiWZJ9/Az55uieaeJCJouh1wOsAklT22lOFGIC91Tne9NwXkzMy5EyLrJkha2TIGXEd9000JQcuIU5F1zR0Lf53M0lhFGL7HrbnYvsuQz+ubc9l6Dlpf5hsH3gOQ89JxwaezTBpD12HQb/PUtHh0PEjDGyboT2kbw8Z2MOz+uhR0zSKuTz5XJ5iLk8hl6eYLyR1nkKuEI/n85TyBYq5AsV8gWI+3q+UL1LI5SkVCkm/gJWxTvkz9x94iBe9/lV87Z47edlP/iJv+o3X8aEb/5M3/O1bOHT8KLvmtvNHL3+tnB8jNrXVjv3AC4FjSqkroii6F3gOZ3AR0MSDzHpUswWec9kTJz0NIcQ5oimNvGmRN099kH+koijC9j0Gns3Acxi4Ttruew4DNxn3HPquzcCz6bs2/aQeeA62rtNoNzlw9BC94YD+cEC338cP/DOag2mYlPIFSoUi5UIxDjhJe35pkS9986vousaLfuSnuOaqq/nkF25ktlbnX/70/1ApligVilSKJaIo2tIfq4mLTxRFvlLqV4EPKKVCoAn8yukepzbjiZFP2H5p9KlfefOkpyGEEEAckNzAp+cO6bs2Pdem5w6X951hMhZv6zqDpB7SsUIePHyQbr+HphThGfxeNnQjDTblYpFqsUy5WKJSLFFJ2tViKanLVEonxiqlMpViCdMwL8Cfjjgbue+6dNVxpdTta51vcj488YlPjM7BOTIXZM6bckVGCCE2EqVUfG6dYVLPl0//gDHfOPYA137oL+kP+vz29/4kr/nenwCg68Rhp+sO6diDtN11BnRGxY7DUNvu0w9DHj5yiHavS7vXodPvnfZjtEIuT6VYYqpcoVIsL2tXy2WmShWqpQqVUiltV8tlauUqOSsrK0JiQ5AgI4QQExBFEe/4yid442f+helChQ/+/Bt42t7HpNunckWmcsV1P38YhfQcm44zoO30adt9OvaJdtcZ0LJPjPcMjSMLx/nWA/fR6rZp97qnfP6MaVItVZgqVZgqj5dq2q6l7Sq1pK4US+i6vu7XJcRKEmSEEOICW+x3eOXH384N+7/K8y5/Mn/5Q79GLb/mbbTWRVMa5WyecjbPLqbP+vFBGC4LOy27d1K7NUzqTo8jznHuuv9eWt02nX5vzedVSqXhp1apMlWuUq9UqVWmqFWq1MpV6qN2pcp0dYp6pUYum30kfxxiC5MgI4QQF9DnH7qTl3/k/9AcdnnTc3+FlzzpBzfkRzS6plHNFamuY1XIC3za9oCW3aM57NG0e7SGcbtl91gadmkOuzQHPRaCBt9+6H6W2k26g/6az5nP5qhXp6hXppiuTjFdrVGv1qhXpqhXp5ip1pieqlFPttXKVVn5uUhIkBFCiAvADwP+4ub/y9u+eD2X1rbz3p95HY+d2zfpaZ0Xpm4wXSgzXTi784XcwI+Dz7DL0rDL0qCbtHs0Bp14bNil1evywOEDLDaX1lz9UUpRK1eZrtaYqdWYrtaYrtaZnqoxU60xM1VPxuvMTNWpVyT4bFYSZIQQ4jw72F7gZR/+K75y6Nv8/OOfyZ8958UUMvJRyUoZ3WCuWGWuWD3jx7iBz9KwS2PQodHv0Bi1k7I46NAi5J4H97PYupVGu7XqSdBKKeqVahxwpurM1qbT9kytzmytzmwyPlubppDLn8uXLh4BCTJCCHEeffSeL/PqT7yDIAx5x4++kh//ju+d9JS2lIxusK04xbbi1BntH4RhGnwW+20Wk7Cz2G/TGHRY6LdpBgHfuPdbLDQbtLqdVZ8nn80loeZEuBn1t9Vm0v5cfZpivrAhPz7cKiTICCHEeTD0XP7wU+/mH7/2Ka7ZfinveOFvcsnUtklP66KnaxozhQozhQrM7D7t/o7vpQFnYdBmod9msR/XC/0WC702Dw0PcusdX2OhtbTqas8o9MzVp5lLQs5cfZpt07Nsq8+k9WytTu58vOgtToKMEEKcY/csHOTaD72NuxcO8htPfQG/9/0/S0aXX7ebkWWY7CjX2VGun3bfIAxpDDrM91ss9NtpvdBvMd9r0bBC7j/0EF/4+ldotJsnPd7QDWzHlnN1zpL8yxJCiHMkiiL++euf5r/f8G6KmRzv/9k/4JmPevykpyUuEF3TmC1WmT2Dc3zcwGeh3+J4r8XxXpP5XouOM5AQsw4SZIQQ4hxo231e/Yl38NF7vswzLnk8f/MjrzijA5q4OGV0g53laXaWz/4eP2I5CTJCCPEIfeXQvbzsw3/J0e4Sr3/mL/CKp/4ImtImPS0hLgoSZIQQYp2CMOSvvvQh/vzm97OrMs3HfumNPGnn5ZOelhAXFQkyQgixDse6S/z6R/6azz18Jz/2mO/hLc+7lnJW7i0ixIUmQUYIIc7SDfu/ym989G8Y+i5v+6GX8/OPe6bcJ0SICZEgI4QQp/Hvd36OP73pvRzuLFLI5Oi5Q75jdi/vfOGruHx656SnJ8RFTYKMEEKcwr/f+Tle84l3MPRdAHruEF3TeNlTni8hRogNQE6rF0KIU/jTm96bhpiRIAz5i5v/74RmJIQYJysyW0gURYRRhBf6+GGIHyT1qB8GBGEQt6Mg6YdpHYQBQRTij7WDMCSIQsJRHUVJHY+Fyc8cjUURRJwYD5PHjOYGUdqPkjlHRMnj4v/Fr2XsdXHyLb9HFCfOSxidoqBG/1NJW8V7KaXQlALietTXlJbUCoWGUqApDX00rmnpfqMxXWloWtzX01rHGGvrmoYxVhsq2a7pcV/TMDQjrvW4NjUjnZuYPNt3OdRZXHXb4U7jAs9GCLEaCTJnKYxCHN/H8V3swMP1PRzfww48HN+NtwUuru9j+y5u4OMk+43aXjrm4wZ+2veCpB/6OL6PH57Y7oVBut1P2n4YrOiHpzzob0SjILIydMRj6qT9xo2/1tH3m6wMRyv32wwUKg41uhGXJPiM98frTDKe0Q1M7UTf0s10PGOc6Fu6mYyZWLqJldTxPhksw8AyMmSTbRnDTNrxtovl/igPLB3jJde/dc3tO8/glvVCiPNvSwQZN/AZeg6DpAw9h6HnxrXvMEjatu9i+6Nxl6HnYnvJWLJt1Ld9D9t3cZJtTtJ3A/+czNnQdDK6gWWYqx58TF0no5tkAihoFhnDiA9mY8XQdDKjg9x4rcZXAEbv/E+M60pLVw7SvtLQknFD6WhavOqgJ+P6aBVixcrEaHVCKYXGytWNFXWyz3houVDGw03IiRWiiOX1aHUpSvYZrUItW4lK2kGy4hREo1WrKFnpivfzk3F/fKUr6fujOhz14xUyb6zvhSeKH57cd0MfLwjwVcjA6+MFwUmh2A28pI4D77mQhh/DJGdk4tBjZMiacfjJmnE/l4znzHhbzsiQM61kLK7zZoacYZHPWORMK26b8fa8aWFO6PuJPvStL/LqT7wDU9f59e/6Ef7h9v9c9vFSzsjwB8/4uYnMTQix3KYMMncvHODqv3pZGl7W8wva0PQTv2yTX7zpL+NQUTULZHNTWLpBTs9g6QbZtE7eySbBI5u0M+mYQUYba4/1M5pBRtcvmne1G4VKAhUKLtZvMgmjEDeIA5AT+LjhaJXwRD9tB3HbDjycYDSerDwGPnbg4gQ+w6S2kxVKG4+W3U/fNNjeiTcJ6/13mjetNNjkTYt8JkthvG9mKWTGyhr9YiZLMZOjaOXW/AJH23d5/af+iX/46n/xnTuv4J0/9ip2lqd57Ny+5KqlBjvLdf7gGT/HT179fY/0P4kQ4hzYlEEmr2f4ge2PIZ8EkVG9rK2fGMvqZrotq5tkjQymdrEezsTFSlMaWUMjizmRn++FQRp4BqNV0MCLV0d9l2EQ16NtJ2onbfeTdjcccrzXjFdhXYe+ZzPwnDOei6npFK1cHGySgKMpjbsXDtBxBjxu7hK+Z89juP5bX6SYyVKy8vz5D76EopWjZOUpZXI0Bh1KVl6+1VqICduU/wL3lKb5X0/7hUlPQwhxFkxNx8zkKJE7L88fRiFD32PgO/Q9h77v0PNs+p5DbzTm2el4zxttt3mws8C328eAiFquxEOt4/zVlz58RudXZQ2TUiZP0cpRtvJpKWVPtCvZPKVR2ypQzuYpWwUqyT6T+ghNiK1A/vUIIbYETWkUTIuCaTFzhlnJCTz+x1c+yCcOfIMnzVzCO77/Jewq1oD4vKqB79DzHLqeTdcb0vccOu4w7Xfd8dqOtwUei82jdJwBHWdA1xmedh5500oCT4FytkDFylNN2wWquQJlq0A1G5dKrpi2i5mcXOUmLmoSZIQQF6WHOgtc+9m/45uNg7z8O57NHzzphcs+clZKxefXmFnmqKz75wRhSM+z6XhDuu6Qtjuk68V1Jyltd5DUQzrugIV+m/2NI7SdPm27n9y6YHW60uJQkytSzRaTuhC3swWmciWmcvH4VLYYt7NFqrkChnzELrYACTJCiIvOxx76Gq/+wj+jK41/fNbL+cE9jztvP0vXNCpWnoq1vi+UDKOQvufQcge0nUFcr2g3nbjfdgc0Bh3uXzpCa9ijbQ9O+fFY2cpTS4JOLV+mlitSy5XG2mXq+dKyMfkYTGw08jdSCHHRcAKPN952Pe+6+yaeOL2PdzzjJewubuz7wWhKo5TJUcrkznquYRTSdoe0nD5NZ0DL6dNyByw5PVrOgKbTj4vdZ7Hf5r7FQzSGXfquveZzlq089XwccOr5MrVciel8ORmLy3i/kMk+0j8CIU5JgowQ4qLwcHeRa2/6O77ROMDLvuPZ/METf3TLX3GkKY0pq8CUVeCSs3icE3g0nT5Ldo8lp8+S06dhd2k6fRp2jyW7R8PucbjT4I5jD7E4aK95j628aZ0UbmYKFabzZaYLlaQd1/V8ecv/NxHnnvyNEUJseR9/+Gu8+vP/glLw7me9jOftefykp7ShWbrJtnyVbfnqGe0fRRF936Fh92jYXRaToLNod2nY3bQ9329x98IBFvsdnMBb9bmq2UIabGYKFaYLFWYLFWYK1XRs1M6b1rl82WKTkiAjhNhyPnD/rbzpqx/hcH+JgmHR8x2umd7HO77/JewpbeyPkjb7Y5L9AAAgAElEQVQjpRRFM0vRzLK3NH3a/aMooufZLNrduAzj8LNod1kYdliwuywGQ+5eOMjCQ3fQsvurPk8xk2OmUGG2UGW2WGU2CTizxSpzhSpzxSnmilWmCxU5sXkLkyAjhNhSPnD/rbz2i//KMIi/UqDnOxhK45ev/D4JMRuEUio97+eS8uxp93cDPwk5XRbsDvPDDovDDgvDbtwObe5ZOMjND91Be5XQo1BMF8rMFafYVpxK622lKbYVa2wrxWMzEng2JQkyQogt5U1f/UgaYkb8KOR/fv3j/Mzl3z2hWYlHIqMb7ChMsaMwddp9bd+Lw86gw4Ld4figzfHhqG4z32/xzWMPsjhon3RZu0IxU6iwvVRLQs4U20s15pKws71UY3uxxlSuKPfu2UAkyAghtgw/DDjUX1p12+E1xsXWkjVMdhfrp73Cyw8DFu0uxwbtNOQcG7Q5NmhxbNDmUHuR2w7dS2PYXfVnbCvW4mCTljrbSzV2lGvsKNWZLVZldecCkSAjhNgS5gdtXvbZv19z+85C7QLORmx0hqaf0QnNTuAxP+xwbNDi6KDNsX6Lo4NWHHj8Pl89up9j9y5h+8tPXtaUYrZQZUc5Djg7y9PsKMUhZ3u5zs5ynW3FKbkvzzkgf4JCiE3vi8fu5eWf/Xu6ns2Lrvw+3r//lmUfL+X0DK974gsmOEOxWVn66Vd4oiii6fQ5NmhxZNDiaBJ2jvSbHBu02N84wmcfvIOeu/zrKhSK2WKVneU6O0pxuPnrP/oxDEMOzWdD/rSEEJtWFEX89Z038KavfphHlWb5t+e+kkdP7eAps5emVy3tLNR43RNfwE9c+pRJT1dsUUopatkitWyRx9R2rblf1x1yJAk4R/tNDvebHBk0OdJv8u3FQ3zl8L38rYSYszbxPzGlVBa4GbCI5/PvURS9YbKzEkJsdG1nwCs//0/858Fv8oJ9T+StT/tFimZ8F9mfuPQpElzEhlPK5Lgyk+PK6vZJT2XDUErpwG3A4SiKflgpdQnwPqAO3A78UhRF7qmeQzv/0zwtB3hWFEWPB54APE8p9dQJz0kIsYHd0TjIcz/2Zj596E7+5Ck/xTu+/yVpiBFCbCq/Cdw91v9z4H9HUXQZ0ARecronmHiQiWK9pGsmZe1vORNCXLSiKOI9936BH/74/8QLfD70317DSx/zTLkUVohNSCm1C/gh4O+SvgKeBfx7sss/Ai883fNM/KMlSJeWbgcuA/4miqJbJjwlIcQGM/BdXvfl9/Fv+7/M9++4ir95+ouZzpYmPS0hxPq9DfgdYPQPuQ60oigafXHXIWDn6Z5kQwSZKIoC4AlKqSpwvVLq6iiK7hzfRyl1LXAtwC65jFKIi8oDnXleeuM7ubt5hN96/PN5zeOfj65NfEFZiK1r6KHuOPpIn2VaKXXbWP+6KIquA1BK/TAwH0XR7UqpZzySH7IhgsxIFEUtpdSNwPOAO1dsuw64DuDx03vloychtrDx70qqWUV6nk3etHjPD/w6z9r1HZOenhDizCxGUfTkNbY9DXiBUur5QBYoA38JVJVSRrIqsws4fLofMvG3NEqpmWQlBqVUDngOcM9kZyWEmJTRdyUd6i8RAQ2nhxsGvObx/01CjBBbRBRFr4uiaFcURfuAnwU+E0XRLwA3Aj+Z7PYi4MOne66JBxlgO3CjUuqbwFeAG6Io+tiE5ySEmJDVvispIuIdd31mQjMSQlxAvwu8Rim1n/icmXed7gET/2gpiqJvAtdMeh5CiI1hre9Eku9KEmJriqLoJuCmpP0AcFY3gZp4kBFCCIgvrf67u29c894L8l1JQojVSJARQkxc33N47Rffw/UP3sbjaru5t30MOzjxJXzyXUlCiLVIkNkEgjDECX3cwMcLfZykdsMAL/Bxl7UD/DDADX38MMAbK/54HcXtIAzjOgrxwwB/VIchYRSm/VE7iEKCMKmjkDCK0rGQiDCKiKIoHQ+JiJL9IiCMIsKxfhSFcU1ElLwVH2+P+mtRnLgRmlJxf3RvNJVsVUpDU3FbW9ZWKKWhK4VGMq4UmlJoKHRNQ0/211XcHo0ZSkPX9LStKQ1D0zA0HUPFta40TE1H1+LaVDqGpmNqJ+oTxVje1w0ymk5GM9K2qRlYuoGpGWR0A0sztsQlyA905vmVz1zHt1tHed0TX8BvPPa5XP/AbfJdSUKIMyJB5iyEUYgd+Ax990QJ3GV9O/AYBl7atn0PO3BxAh/bd7EDHyfwsAMvrd0gDidu6KXb3cBPw0sQheftNZljB97RwdfQNHSlJ7U2NpYcsMcO6OZUCWvU1+LtmqbQlJaEguVj42FhtB3iL11TSqXBRCmW3a11PLCMjAecKIrDz2gsSgLV6L/bKFyN6jAKCcNRqBptj8fioBak271mFy/0CMI4zIVJiPPHAp2fhsG4H4fEOBh6YXDe/vvpSktDjaWbZHSDrG6Q0Uws3Uj6JpZujtUGWSODpRtk9QxZwySX1vF+OSNzouiZZf2sbqT/3R6p/zzwTf7fz70bU9N573NewTN2PgaQ70oSQpy5LRlkgjCk7zv0PJu+59D3nbQerOj3PYeBn2zzXQa+w8Bz6fsOQ99lkIYUh+HYUvfZMDWdbHqQyZBLDiiWbpCvVymYGSzDxDIzWEYmrk2TjGEm/bg9KpaxvJ8xDEw9rjOGiaEbyVg8bhp60jYw9NG4jq7pcmv3CyCKIoIwwAviVTM/8PFGxQ/wAi9eTfPjba7v4fo+XhDXcT8uzqj2XLzAx/E8HN9NxuJxx3eT2sN2HTpLbZygix3EQdoOXNzAxw68dYesUcDJJ+Emb2TIGxZ5M64LhpWOFcy4P6rzpkVWN/ngA1/hffu/xKOndvL2p/8yV1Tki/SEEGdvUwaZQ70GL7vpXfT8OKj0PJueNwou9lkFjoxmkDcyFEwr+aUc13M7tpO3cuQzFnkrRy5jUUjrLLlMlpxpkbMscpks+UyWbCYT16ZFLhOXrJnB0DflH7M4R5RSGEmIzGFNejrL+IHP0HWxPYeh62B7DgPXxnZdBq7N0LUZOsmY59B3bIauQ98ZMnQdBs6QQVJ35xssDLsM/cXkDUL8RsAN/VPO4e7mYZ754T8FIKebFMwsRTNL0bSSOi4lM0vBtCiZOUpmlmLmxHjJzFLKxOPlTI6CYUlIF+IisSmPsB13yB1LB9Nfdrv37KGULVDI5ihm85SyeYrZPAUr7hetHMVcnnwmSzGbo2DlKGRzFDI5TGNT/hEIcU4YukEpZ1DK5c/bz3jvFz7JH3/wnRxpLlIrlvECn4Fr8+LvfwFPveyx9JwhPXtA3xnStQf0xkr72ALzwzYPdObpujY932bou6f9mZpScdgxc5QzWcpmjnImLqVMLu1XMvmkzlHO5JfVGXkDIsSmsCn/pV6953I++4Z3TnoaQojTeP+XbuDV//y/GLoOAI1eGwW87oW/wu++4JfX9Zx+4NNzhnSHA7rDPl27T2fQp5PUaX/YpzPs0R0OaA97HJtf5Nuto3Q9m447PO25ZzkjQzWTp5IEm4oVt6uZPFWrQMXKpdunrAJVq0DVircbmr6u1yaEOHubMsgIITaHN37wujTEjETAP3/u4+sOMoZuUM2XqObX/83XURTRs4d0hj3ag7Ey7NHqd5N+l9agR2vQZenIMQ73mtzlHqbtDuh59imfv2zmqFpxwKlZRWrZAlNWgamxdt0qUssW0+2Wbq779QhxMZMgI4Q4bw415s9q/EJRSlHK5Snl8uyszZ714/3Apz3o0ex3afW7NPsdWoO4bva7LPXaLPU6LPU6LB47xgPdeZbsHt1TBKCimaVmFdJwU8+OSimurSLTSXs6V5LzgIRISJARQpwXt+6/E6VUehn8uF31sw8PG4mhG9RLVeql6lk9zvU9mv0OjW6bpX6HRrdFo9eO+702jV6bxW6LhaPH+HbrKEt2d82LFyzNYDpXSsPOdFJmcuPtctzOlTDl4y6xRUmQEUKcc/9w00f47fe8jVqxTNce4HgnTtDNZSxe/+PXTnB2k5MxTOYqdeYq9TN+TN8Zsthtsdht0UjqhU4rGWuy2G1x7PAR7m0dY3HYwVnjKrEpq7As6Mzmyszmykwn9WyuzEy2LKFHbDoSZIQQ54zjufzOv/4l7/7sR3n21U/h7659PZ+64xbe+MHrONSYZ1d9ltf/+LX89Hc/Z9JT3TQKVnyl5d7p099nZ3Tuz0K3yUJnifl2M2k3mW8vMd9pcvzgIe5oHGTB7q55rk/NKjKXT8JNrsxcrsJsUs/lK2zLV5jNVSiYG+t2AuLiJEFGCHFOHG0u8v+8/Q+59f67ePXzf4E//PGXoms6P/3dz5HgcoGMn/vzqNmdp91/4NjMd5ZY6DQ53l5ivrPE8VYjDjztBkcPHmJ/e56FYWfV+wGVzVwabObyFbbnq2zLV5fVM7mSXMUlzisJMkKIdXn/l25IV1pmylVszyUIQ979a3/Ej33nMyc9PXEG8laWfTM72Dez45T7RVFEs9/leLvB8fYSR5sLHG0tcqzV4EhrgSMHDvKlY/s5Pmjhr7isXVOKuVyFbfkqOwonQs6OwhTbk3pbviJXbYl1kyAjhDhr7//SDbzyH/8ivbR6vtNEoXjDT/yqhJgtSClFrVimVizz6J2XrLlfGIYsdlscaS1wtLnIkeZiGnoON+e57/Bhbj5yz6pXb01nS0nQmWJHIS47k3pHforthaqcuyNWJUFGCHHWVr8/TMS7bvowr/6hX5zQrMSkaZrGbKXGbKXGE/ZeueZ+3eGAo60FDi8tcKS5wOGleQ435znSXODAwQN8+fh9tN3hsscoFHP5MjsLtTTk7ErbNXYVa9SsglySfhGSICOEOGsb9f4wYnOIz+PZyxXb9665T88ecHhpnkNL88vqA/c/wLeWDnHDwTuwV1yanjMy7BoLNruLdXYX6+xJ6tlcWYLOFiRBRghxVr720LfRNEUQbr37w4iNo5jNc+WOfVy5Y9+q26MoYqnX5uDScQ415jnYOMahxnEONI5z4OGHuOPAQRp2b9ljsrrJrkIScEpxwNlbmmZPcZo9pTrVTF6CziYkQUYIccb+/cuf4hX/8GYquSJ915b7w4iJUUqlNyVc62OsgWNzYPEYBxpHeXjhKA83jvHwwlEeeugBvt54mKbTX7Z/ycyytzSdhpsT7Tq7ijU5IXmDkiAjhDitIAx44wfeydv+41/5nisexz/9+h9z4123yf1hxIaWt7JctXMfV+3ct+r29qDHw4tJyFk8mrbvffhBPnXwzmU3F1QodhSm2JeEm7ieYV9phn2laSrW+fsGeXFqEmSEEKfUHvR46XVv5L+++WVe/IwX8Bc//5tkDFPuDyM2vUq+yOP2XM7j9lx+0rYwDDneXuKhhSM8tHCEB5P6/v338V8H72DR7i7bf8oqpKFmXzkOOJeUZrikPMN0tiQfWZ1HEmSEEMuM3x9mrlonCkMa/TZv/aXf4iXP/NFJT0+IC0LTNLZPTbN9aprvvuJxJ23v2QMeWjjKg/OHeXDhMA/OH2H/t7/NbQsP8uGHbicc+46xopnlUeVZLi3PxnVlLu2XMrkL+bK2JAkyQojUyvvDHGstAvDaH/4lCTFCjClm81y9+1Ku3n3pSdtc3+PA4jEemD/EA8cPc//8Ie67+x5uX3iQDz14OxEnQs5srsyl5TkurcxyaXmO33dfQiaTuZAvZdOTICOESK12fxiAf/vSf/GHP/6rE5iREJtPxjC5bNtuLtu2+6RtjufGqzfHD3LfsQPsP3aQe+/9Nv9x4BvYvscbTDmh+GxJkBFCpOT+MEKcX5aZWfME5O5wIOfSrIM26QkIITaGxU4L01j9vY3cH0aI86+Ukyuf1kOCjBCCuw7dzzP/+FrCMCRjLF/alvvDCCE2MgkyQlzk/uPrX+C5f/rruL7Hf/3B2/mbF/8eu+tzKBS763P81Yt+Ry6zFkJsWHKOjBAXmfHLq8v5Au1BjyfsvZL3vvLP2DE1w5MuebQEFyHEpiFBRoiLyMrLq9uDHrqm8avPeiE7pmYmPDshhDh78tGSEBeR1S6vDsKQN3/k3ZOZkBBCPEISZIS4iBxqHF9jXC6vFkJsThJkhLhI3HjXbbDGPSrk8mohxGYlQUaIi8C7P/tRfuJ//zbbK3Wy5vLbn8vl1UKIzUxO9hViCxq/MqmYzdG1B/zA1d/FP/za/+CTX/9ium1XfZbX//i1cpWSEGLTkiAjxBaz8sqkrj3A0HR+6rueTTlX4Ke/+zkSXIQQW4Z8tCTEFrPalUl+GPAnH3rXhGYkhBDnz8SDjFJqt1LqRqXUt5RSdymlfnPScxJiM5Mrk4QQG51SKquUulUp9Y3k2P9Hyfh7lFLfVkrdqZT6e6XUab8OfOJBBvCB34qi6DHAU4FXKKUeM+E5CbEpyZVJQohNwgGeFUXR44EnAM9TSj0VeA9wFfBYIAe89HRPNPFzZKIoOgocTdpdpdTdwE7gWxOdmDhJGIZ4gY8X+PhhQBAEadvzA4JwVEKCMMQPA8KxfhiN6oggDIiiiDAKiSKSOiKMIqIoIiICIIqSepX5jA7Xo6+9Vyg0TaFQKKXQlIZSJLVC13Q0pdA1DU1p6FpcNE3H0PS0r2s6uqZjGvG4qRvo+om2qRto2kZ4D7D8pN5qoURr0GVHdZpGr43tuel+cmWSEGIjieJf7r2kayYliqLoE6N9lFK3ArtO91wTDzLjlFL7gGuAW1bZdi1wLcDu+twFndekBWHAwHEYujYD18Z2XYaeg+06cd9zGLoujudiey6O5zD0Rn0Hx3NxfC+uPQ/Hd3F9D8fzcP2kBD6u5+IGPp7v4fo+buDh+z5eEljCKJz0H8WGoSktCTU6pmFi6gYZw8A0TDK6QcbMxLVhkjFMLDOpjQyWaWKZGSwjrrOmRdbMkDUzWEnbMjPkMhlymSw50yKbsciZFjnLIp/JkstYfOyrn+NV//SW9HyYZr+DpjR+9wW/TC6TlSuThBCTNq2Uum2sf10URdeNOkopHbgduAz4myiKbhnbZgK/BJz2dJMNE2SUUkXgA8CroijqrNyevPjrAK7Zd9Vqb9AnLgxDes6Qnj2gZw/oJnXPHtC3h/ScIX17SN8Z0nfsuLaH9N0hA8dO2jYDZ8gwCSlDx8Hx3dP/8DVYRnyAzJhmXBsmlmGSSQ6o+v4eJQwy6JgUMNGx0DHQMNExx9orax0NY6zoaVHL2trYmEIlYyrpraxJesCy/ycdBU6s2KwYiYgIT2rHdUhEkPQDQgIiwqSO+3HbJyQkxF9WgrTtEeBHIZ4f4PshrhOk2x0CvKQ4uHgMsS8r0h70cP2xQLkiWJ4rYRTy2ve8je+76hqesO9KnnbFEyhkc9xxcD8PLhymYOXIW1mKVo5CNkcxm6eYzVNK6mI2T9HKbZgVJyHEZIStAf2PfPWRPs1iFEVPXmtjFEUB8ASlVBW4Xil1dRRFdyab3w7cHEXR5073QzZEkEmS1weA90RR9MFJzMHxXDrDPu1Bj/agS2vQozPsxyVt9+gOB3TsPt3hgO6wT3es3XOGZ/zzchkrPagUrBz5TBbrwSFzZMhTIYdJngw5zGUli5HUcXu12sLASsKJ5qv4LKQzn5o41/afenNIhEuAg4+Dj42HvUY9TNpDPP6EG1Z9Ptf3aPW7HGku0HeSkJyE4zNVtHKUcgVKuTylbCFtl7MFyrm4X84VKOeLcZ0rUM0XqeRLVJIxa8WN94QQYjVRFLWUUjcCzwPuVEq9AZgBXnYmj594kFHxCQ7vAu6Oouitj+S5bM+h1e/S7Hdp9js0+x1a/V5Sd2kN4tIe9NJ+HFx6y84nWGOelLPJL/PkF3npQY8dlChSp4RFAYsimbQuYlEgQyFp55N2HhPd1cAFuo/kFYutQEMlIfTM/zkeocOb+TQ+J3/ct5MKH37wp08aDwgZ4NHHZYBLD4feinZ/VDsOXceh13Lp4jC8zGa+3UjDfdcepOcvrSVrZqjki1TyRar5EtVCiWoSdEb9qUKZqcLy9lShLCFIiC1OKTUDeEmIyQHPAf5cKfVS4AeBZ0fRmZ3PMPEgAzyN+HOwO5RSX0/Gfn/8hJ+VDi0d50VvfwNL/TbNXoelXptmv8vAtdf8IUqpE79A8yVKD3tcQZESdSrkqJClvKxYlMbqQpRBGypZ2RATcT138GY+zRHazFDEwU8+otNxCdL9cpj8Hs9e9Tl0NEpYlLDOfgIrVpVCoiT8OHRw6GDTxaaNTTfptz2bdntIp+3QZsD8Xp97jx5I30CcKgjlM1lqxXIcboplaoUK9VKFejEu06Uq9VKVeilpFysSfoTYXLYD/5icJ6MB74+i6GNKKR94GPhSciHHB6MoeuOpnmjiQSaKos8Dq18vuoZmv8tdh+6nViyz7aDGo9nJFDmqK0qFbFLnKEUWWl9B/zy9ECHOk+u5g9/howzxAJhPTvT/XZ7FLqppwNlBhd/j2fwYjz3vc9JQaejfcaYPevhEMySii0ObIa2ktLFpMaTJgJY7pLVk01wa0LnM5a5D99PotWn2O2sGoEquyHS5ynSpykxpiulyXM+UT5TZ8hQz5RpThVJ6tZsQ4sKLouibxBf3rBw/61wy8SCzHt8RzvKJYy+e9DSEuCDezKfTEDPuX7idW3jVBQku55qGokKWCln2MHXqncdWgwJC2tg06NNgQIM+TQYs0qcxHLA0HNCsZXhg/hC33n8Xi93WqlfbGbrObLkWl8oUs+Uac5U6c5Uas5Uac5W4v61ap2DlzvGrF0KcS5syyAhxMTlC+6zGtzIdjRp5auS5fK2d7j7RDIloMWSBHov003ox6LPQ7LHQ7HN87xJ3HNjPfKdJEAYnPV3RyjFXrbOtUmeuWmd7dZptSX9bdTrpT1PK5c/LaxZCnJoEGSE2sIiIAhY9Tr7iaAeVCcxoc9FQafC5cq2dko+8RqHnOF0W6DFPj+N0mXd6LBzv0ajANx6+l09+/Yurno9XtHJsn5ph+9Q0O6pxvb06zY6pGXbUZtg5NcNMeQpd08/b6xXiYiRBRogNZvzE3hwmAzx0FMHYXXNOdVKvWJ/x0PNoVrnp5r1xFRHRw+U43bQco8txp8uxY10WCg5fuPfrHG0t4gfLV3gMXWdHdYadtVl21WfZXd/Grtocu8fasrIjxNmRICPEBrLyxN4BHgYaP8cT+Qz3XfCTesXJFCq9+usypk/e4f64Colo0OcYXY7S4RgdjgQdjjQ6HK8rbt1/F9d/5caTwk61UGJ3fRt769vYXZ9jz/Q2dte3sWd6G3umtzNVKF2AVynE5iFBRogNZLUTe31CPsN93MKrJjQrsR4aihmKzFDksWxfvjFZ3QkImafHYdocosVh2hzutznUb7PfP8iN37qN/oobbVZyRfZMb2Pv9Pa03juzPa3l5GRxsZEgI8QGIif2Xlx0NLZTZjtlnszu5RuPxB9jNRlyiBYHaHGYFgeGLQ4dbLE/OMhn7vrKSefrzJZr7JuJQ82+6R3sm9nBJbM7uGR2J9sqdfn6CbHlSJARYoN4iCW0FefCjMiJvRcnNXbezuNW3rEnCToNBhygyUFaHKDJw50mR3Yqbtl/Jx+45TPLLj/Pmhn2zcTh5lGzO+Myt4vLtu1mV21WTkQWm5IEGSEmaPzE3tEXahpoOGd4t15xcVMopikwTYEnsuvEhuQSdI+AQ7R5mKU45HhNHjrS5KB2jM/d87VlH1tZRoZLZndw2dxuLtu2m8u37eGK7Xu4bNseasXyBX5lQpw5CTJCTMjKE3sjIkx0foYn8Gk5sVecAyY6l1DjEmrLNxyK/74dp8dDLPEADR7wGzx4pMG90cP85ze/hBf46e71YoXLt+3h8u1xuLli216u2LGXvdPbZBVHTJwEGSEmZLUTex18Pi0n9ooLQKHYRoltlHgqe09sOBqfYH6AJg/Q4H4Wub/X4GEt4JPf+CL//LmPp7tmDJPL5nZx+fa9XLVjH1ds38uV2/dy+fbdZM11fKeXEOsgQUaICTksJ/aKDcpA41HUeRR1foAr4sHkSqsmQx5gkftYZL+/yP7Di9zh3cdHb785PR9HUxp7Z7Zz1Y69XLl9H1ft2MdVOy/hyu17yVvZCb0qsVVJkBHiAhk/H6bK2pfIyom9YiObIseT2M2Txq+ymgcbnwdosJ8F7o0W2D+/yP3mUT51x63px1RKKfbUt/HonZfwmF2P4urdl3L1rku5dG4Xhi6HI7E+Z/Q3Ryl1FbATuCWKot7Y+POiKPrk+ZqcEFvFyvNhmgxRgI7Clzv2ii0gi8FjmOMx43dFPhyfcPwQS9ybBJz7Fhe4zzrKp+68Jb0ZYNbMcNWOS+Jgs/tSHrvncq7efSnVvNz8T5zeaYOMUuqVwCuIz4N/l1LqN6Mo+nCy+c8ACTJCnMZq58NEQIkseTJyYq/Yskx0LmeGy5nhh0aD/397dx4cZ33nefz91dFS67QOS75kyZJs2bINNra5wxFIlhzA5qqETMJMSC21uzO7IUltNiSbZOeoLDvZmk1qZ6Z2nISlsmHJBc4xQEjCkFAchoCB2MYcBl+ybMuyJOtWS93f/aNbimysw0LS04/686p66unup/vR9ylb7o9/13M0OR5sP+3s4wQvDx/ntcIRfvnSU3z/iYfGPruyYgkbVzayoaaRjTWNbFzZSG3lUswskGuR9DSdFpl/A2xx914zqwN+YmZ17v4tQH+bRKZhonEvXQywmy/MczUiwcsjh/UsYT1LgAvh5eRMqjZ6eZkT7OU4+06dYF/kMA+9+CTuyZbL0oIiLli5+oxtzdKV6prKYNP5k88a7U5y94Nmdg3JMFOLgozIpEbHxbx1ibskjYcR+SPDqKaYaoq5lsbki8egnxiv0MZejrO3/zj7hvr57mM/ZXA4BiS7pjbUNHJh7ZrUtpp1y1aRlxsJ8GpkvkwnyJwws03u/iJAqmXm/cDdoDZwkYmcPZ8vPkAAAB5mSURBVC7mbBoPIzI9BUS4iBV/XPTvQHKK+Bu0s4fj7B0+xr7IED/e+Wu++9hPgeTU8A01DWytb2bLqnVsrW+moXqFuqUWoOkEmVuBkfEvuPsIcKuZ/dOcVCWyAJxrXMyo5RoPI/K25JBFE1U0UcWHuABeSd5x/BCd7OEYL420sifSz71PPMz2Rx8AkncWHw012+qb2VLfrFWLF4Apg4y7twCY2beAO3y0ozJ57Mk5rE0kdMZPsZ6oO8lAC96JzIEsbGwl4xtZD68m7zD+Gid5gaO80NfC7q5TfOMX3xtb86axuoZtDc1sbVjPtvpm1q+o13ibkDmfP60e4Odm9jF37zOzfwV81d2vmKPaREJlqq6kURoXIzJ/ssliHdWso5qPcxG0QB8xXqKVXbTwwokWfjPwLPc99QgABZF8ttSvY1vDei5p2MC2hmYqihcFfBUymWkHGXf/L2b2ceC3ZhYDeoEvzlllIiEzWVfSKI2LEQleIREup47LqQPAu50jdPE8LeyKtfDiYDffevg+4onkOjcN1Su4pHEDFzds4MqmTTQuqdFYmzQy7SBjZteRnIrdBywFbnP3V+eqMJGwGO1OmuiWA5DsTtI6MSLpyTBWUsZKypK/nwdhgGFeopXnOMKuEy080vc0/+/J5LJpVSXlXNF0IVc0beLKpk2sXVanYBOg8+la+jLwFXd/wsw2Aj80s8+5+7/MUW0iaW863UnLKdWYGJGQiZLLpdSO3VDTe503OcVODrGz+xDP7t/Djt8/BkBl8SKubNrMlWs3cdXazaxZWqtgM4/Op2vpneMe7zaz9wD3A5fPRWEiYTBVd5K6kkQWBsNooJIGKvkTtuCdzmG62MlBnu45xM439/LT55LBZnFJGVc2beIday/i6nUXadr3HJvx0Gx3P5bqbhLJWJPdqVpTrEUWLsOopYxayvgom/GOZLB5moM81X2QneNabJaVLeYdazdz1dqLuGrdRaysXBJw9QvL25pj5u4Ds1WISJhMtWKvupNEMsv4YPMxNuOdzgE6eIqDPNV5gEf3PMsPn/4VAHWLl3FN8xau33AJVzdvoSRaGHD14abJ8iLnSSv2ishUDKOeCuqp4BNswXuc1zjJkxzgyZMHuP+ZR7nnd78gJzubSxs3cv3GS7h+wyVc7leqG+o8KciInCet2Csi58uwsZWIb+MShgfjPMcRfhvfz+P9J/ivP/kn/mbHd2i/44OUlmqtqfOhICMyTVNNs9aKvSIyXblkcxl1XEYddx6B4/SwJ35MIWYGFGREpmE606y1Yq+IzNQSillCcdBlhJKCjMgkprPYHWhcjIhIUBRkRCYw3XsnaVyMiEhwFGREJjCdeydpmrWISLCygi5AJB3tYLe6k0REQkAtMiJnGe1Smoy6k0RE0oOCjMhZJutSipLL33KjAoyISJpQ15LIOFN1KSnEiIikl7QIMmZ2t5m1mdmeoGuRzDVVl9JyShViRERmgZnVmNljZvayme01s8+cdfzzZuZmVjnVudIiyAD3ADcEXYRkrh3s5g526P5JIiLzYwT4vLs3A5cCf25mzZAMOcC7gcPTOVFaBBl3fxzoCLoOyUyjLTHxCe9lrS4lEZHZ5O7H3H1X6nEPsA9Ynjr8P4EvwCT/KI8TmsG+ZnY7cDskm/hFZsNoS8xkIUZdSiIiM1JpZs+Ne77d3bef/SYzqwM2A8+Y2c3AUXd/abp3AQ9NkEld/HaAC23ZtFKayGSm0xKjLiURyUQ+PEK8tf3tnqbd3bdO9gYzKwLuB+4g2d30JZLdStOWFl1LIvNtqjExANmYupREROaImeWSDDH3uvsDQAOwCnjJzA4CK4BdZrZksvOEpkVGZLZMtyVGIUZEZG5Yst/ou8A+d/87AHffDVSNe89BYKu7T9o0lBYtMmZ2H/A00GRmLWb26aBrkoXrKzyslhgRkWBdAXwSeKeZvZja3juTE6VFi4y73xJ0DZIZdrCbLgYmPK6WGBGRuefuTwCTjuZ197rpnCstWmRE5sPouJiJqCVGRCR8FGQkI0xnXMw3+YBCjIhIyCjISEaYalxMGVGFGBGREFKQkQVvOuNi/or3zGNFIiIyWxRkZMH7Cg9PeEzjYkREwk1BRha0L/HgpK0xGhcjIhJuCjKyYO1gN/+X5yY8rnExIiLhpyAjC9ZXeHjSW6dqXIyISPgpyMiCNFWXklpjREQWBgUZWXCm6lIy1BojIrJQKMjIgjNVl9In2arWGBGRBUJBRhaU6XQpfZ33zWNFIiIylxRkZMH4Eg/yPXUpiYhkFAUZWRCmGhcD6lISEVmIFGRkQbiLRycdF6MuJRGRhUlBRhaEo5ye8Ji6lEREFi4FGQm9L/HgpMfVpSQisnApyEio7WD3pAN8AXUpiYgsYAoyEmpf4BeTHl9O6TxVIiIiQVCQkdDawW4GGJ7wuAFf5Lr5K0hEROadgoyE1lStMRobIyKy8CnISChN1RpTSERjY0REMoCCjITSVK0xd/H+eapERESCpCAjoTNVawygLiURkQyhICOhM1VrTBnReapERESCpiAjoTNVa4xW8RURyRwKMhIqU63ie6tmKomIZBQFGQmVe3l+0uOaqSQiklkUZCRU4pPc47qQyDxWIiIi6UBBRkIlG5vwmKZci4hkHgUZCZU/Ycs5X7+SVRobIyKSgRRkJFS+zvu4la1jLTPZGLeylR9wa8CViYhIEHKCLkBkuqwkj0X/52b+26fy+Hq3BvWKiIhaZCRE8m9qIvqBdeTfuCboUkREJE0oyEhoFHxqU2q/OeBKREQkXSjISChYSR6Ry2oAiFxegxVrqrWIiCjISEjk39SEx+IAeCxO/k1NAVckIiLpIC2CjJndYGavmtl+M/ti0PVI+in41CaySvIAyCrJU/eSiEjImdndZtZmZnvOev0/mNkrZrbXzP52qvMEPmvJzLKBfwDeBbQAvzezn7v7y8FWJvOt7McfIfqh5nMe86GRM55HrqhhWeJr53zvwP0v0/mRH896fSIiMqvuAf4e+N7oC2Z2LXAzcKG7D5lZ1VQnSYcWmYuB/e7+prvHgB+QvAjJMN13PkrshWMkemNvOWZ5OZM+B0j0xojtOkb3nY/OWY0iIjI73P1xoOOsl/8dcJe7D6Xe0zbVedIhyCwHjox73pJ6TTJMfH8H7du+Tc9XHyPRP4yPJKb1OR9JkOgfpuerj9G+bTvx/Wf/XoiISEisAd5hZs+Y2e/MbNtUHwi8a2m6zOx24HaA5ZQGXI3MmYTT982dDP7za5T94MPkrKkgq2jiGUqJ3hgjr7XT+dGfEH+jcx4LFRFZuLKKokSv2vD2TnIPlWb23LhXtrv79ik+lQOUA5cC24AfmVm9u094x+B0aJE5CtSMe74i9doZ3H27u291960VFMxbcRKMZOvMdnrvegLvHz7ne7x/mN67nqB927cVYkRE0k/76Pd2apsqxECyV+YBT3oWSACVk30gHYLM74HVZrbKzCLAx4CfB1yTpAOH+OHTE3Yx+UiC+KEumDCni4hIyPwUuBbAzNYAEaB9sg8EHmTcfQT4C+ARYB/wI3ffG2xVki7GT7v2hJPoi+GJZHLRNGwRkfAys/uAp4EmM2sxs08DdwP1qSnZPwD+dLJuJUiTMTLu/hDwUNB1SHoZv5qvD42Q6Bzk9B2/pPSbN5BVlo/l5Yyt8us9b53pJCIi6cvdb5ng0CfO5zyBt8iITCT/piY8niDRF2P4+VbaNvwjgz/aS9uGf2T4+dZU60xCq/yKiGQwBRlJWwWf2kRWYYT+7+yi/ep78I4BALxjgPar76H/O7vIKoioe0lEJIOlRdeSyLnE2/rouOUnDP7wHEOm4k73Zx8htrOF/JvXzn9xIiKSFhRkJG113XL/lO8Z/OHecwcdERHJCOpaEhERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdAKNMiY2UfMbK+ZJcxsa5C1iIiIyPwxs8+mMsAeM7vPzPJncp6gW2T2AB8EHg+4DhEREZknZrYc+I/AVnffAGQDH5vJuXJms7Dz5e77AMwsyDJERERk/uUAUTMbBgqA1pmcJOgWmWkzs9vN7Dkze+4U/UGXIyIiIpOrHP3eTm23jx5w96PA/wAOA8eA0+7+q5n8kDlvkTGz3wBLznHoy+7+s+mex923A9sBLrRlPkvliYiIyFkSh7rpue3Rt3uadnc/5/hXMysDbgZWAV3Aj83sE+7+/fP9IXMeZNz9+rn+GSIiIhIq1wMH3P0kgJk9AFwOnHeQCU3XkoiIiCwYh4FLzazAkgNlrwP2zeREQU+//oCZtQCXAQ+a2SNB1iMiIiJzz92fAX4C7AJ2k8wj22dyrqBnLe0AdgRZg4iIiMw/d/8a8LW3ex51LYmIiEhoKciIiIhIaCnIiIiISGgpyIiIiEhoKciIiIhIaCnIiIiIBMxxjtEddBmhFOj0axERkUzVyxBPcIDfsp/HK45yqvc0HbG7iEQiQZcWKgoyIiIi8yBOgt0c40kO8MTaDna+vpvh+AhFeVGuWbmV6zZeTDweD7rM0FGQERERmQMJnFdo4ykO8MymHp569Q+cHugFYH1vPf/+3R/hXRsv5ZLGDURycgGIRqNBlhxKCjIiIiKzwHFep52nOMDTHGJnUbK7CKC+dTkfuPharlp3Ee9o2kxVaXnA1S4cCjIiIiIz4Dj7aedpDvIUB9lZ3Ep7TxcANRXVvLvpUq5et4V3rNvMivLqgKtduBRkREREpiGB8ypt7OQQOznEsyXHONndCcDyssVcv/YSrly7iXes3Uxt5VKSN3WWuaYgIyIicg79xHiRVnZxhOdp4fmiE3SkuopqKqq5bs3FXNF0IVc0baK+armCS0AUZEREJOM5zhG6koGFI7xQe5o9R94gnkjOImqsruG9q68YCy61lUsDrlhGKciIiEjG6SPGS7SyixZ20cILJW1j3USFeVEuiq7ls+/9OBc3bGBbQzPlRaUBVywTUZAREZEFLU6C12nnBVp4kaP8oaabvS1vkvAEkGxtub7hYrY2rGdbfTPrV9STk62vx7DQn5SIiCwYo0v9v0QrL3CU3Wv7eeHAK/QODQBQWlDElpJ1/Kcbb2VbfTNb6teptSXkFGRERCSUEjiH6GQvx9jNMfatj/HSodfG1m7Jyc5m4+BqbrniBrasWsfW+mYaqleQlaXbDC4kCjIiIpL2YsR5nZPs5Th7OMYrawbZfXg/PYP9AORm57CuZxXv3XwlF9auYVPtGjaubCQ/Ny/gymWuKciIiEha6WKAlznByxznZU6wb2Ufr7YeJDYyDEBBJJ8N8QY+etm7uaB2DResXE3z8lXk5epmi5lIQUZERAIRJ8EhOnmZ4+zjBPto45WK0xw+dXzsPYtLythY3Mi113+YC1eu5oLa1TRUryA7KzvAyiWdKMiIiMiccpzj9PAaJ3mdk7xKG6+tGmTf0QP0xwYByM7KZvWSGratWM9t197MxppGNq5spLq0IuDqJd0pyIiIyKxI4LRymtc4yX7aeZ2TvNEQ49XWQ2N3fQYoLyplfV49f3b1jaxf0cCGlQ2sXVan8SwyIwoyIiJyXmLEOUgH+2nnDdp5nXbeqB3k9WOHx1pYACqKSlmXs4oPX3o965bVsXb5KtYuq2NxSVmA1ctCoyAjIiLn1EE/b9DOG5wa279Z3c+Bk61jS/dD8oaJa4pq+dOr3s+aZbWsXVZH09JaKooXBVi9ZAoFGRGRDDbAMIfo5E1OjW0HG4Z5/fgROvu6x94XycmloXoFzUvq+dfbrqFpaS2rl9bSWF1DcbQgwCuQTKcgIyKywHUzyGE6OUgnh+jgMJ20rDPebGuhpaMNdx97b3VpOatzVnLz1qtZs7SW1UtqaFyyktrKJZopJGlJQUZEJORGSHCMbg7TySE6OUInh+miZVWcAydb6UitdDuqoqiUVUPLuXzNhdRXLadxyUoal6ygvmoFJdHCgK5CZGYUZERE0lwCp41eWujiMJ0coYsjdNG6zjh08hhHO9sYif9xzEpOdjYryqupzV/KTVuuYlXVclYtXsaqquXULV6msCILioKMiEjAholznB6OcpoWujjKaY7QxfHmbA63H6Olo21sVdtR1aXlrIwtZVvDej5UeR11i5dRt3gpdYuXsbxsse7eLBlDf9NFROZQAqeDfo7TTSvdtHKaVro5ymlONBpHTh3nWNcpEp4443PVpeXUDC5hU10TN265ipqKJdRVLmVl5RJWVi4lGtGaKyKgICMiMmMDDHOCHk7Qw3F6OE53at9D++psWjtPcqyr/S2tKbnZOSwvr6Imp5qrm7eworyamopqVlRUU1NeTU1ltRaHE5kmBRkRkXEcp48YbfRygh7a6OXkuMft63I53tXO8a5TZ6xWO6ogks/SskqWZlVyceN6lpUtZnlZFcvKFrOsrJIVFdUsLi4jKysrgKsTWXgUZEQkIwwwzEl6OUkfp+hLPU5ubfTR0ZhNW3cnbac76BsaeMvnIzm5LCmtYEmsgqZldVzdvIUliypZuqiC6tIKlpUtZmlZJaXRIswsgCsUyUwKMiISSjHinKKPTvo5RT/tqYCS3PqTxxpyaOvupL27k95zhBOAssISqkvLqcotZ2v9OqpKyqkurWBJKqBUlyaflxUWK6CIpKFQBpkDdPA5fkYZUcqIsogoiygY9zi5FZCLoX94RNLZaFfOaQbpYoAuBuhkgC76U/vk8w766arP4VTvaU71dNE90HfO82VnZVNZXMri4jIqIkVsa2imqqScyuJFVJWUU1VaRmVxGVWlZVSVlBPJyZ3nKxaR2RTKIDMSgaeKWuno7T7jBmVny8nOpqywhLLCYhYVFFP4xhClRCkln0XkU0qUEvIpIZ/S1H50KyaPHNSHLTIdIyToYZDTDNLDEN2px8n9AN0M0pV67TQD9Dbk0dXXQ1d/D5193WesgXK2gkg+5UUllBeVUp5fQN3iZVQWl1JZvIjyokVUFpdSUbyIqpIyKosXsaigWONPRDJIoEHGzL4B3AjEgDeAT7l711Sfa1pax+++9m0ABoeH6OjtprOvm47ebrr6kv8wdqb2Xf09dPam/sGsHeFA/ym6+ns43d97xrLc51KYF6UkWpjcCoooeGOQYvLGtqKz9oXkUURkbF9EHoVEyAtnXpQFznGGiNNPjF6G6CVG37h9D0P0nrUf3frq8+ge6KVnoJ/ugb5zjikZz8woLShiUUHyPxWL8qIsL6uirKiYssISFhUUU1aYelxYnAwuhaWUFRVr9o7IAmVmNwDfArKB77j7XTM5T9DfsL8G7nT3ETP778CdwH8+nxPk5+alZgMsPq8fnEgk6B7so6uvh+6BProH+jidCjjd/X2cHuilu7+X06lj3QO9dK9K0DrQS+9gGz0DfRP2uZ8tNzuHwvwoRXlRCvOiFOZHyT8wSCERCogQJZcCcikgMraPjtufveWTQ/64vVqOFq4REgwyzCAjY/sBht+y9RMb2/eP24++1keMwVX59A0O0Dc0QO/QAH2DAwzHR6ZVR1FelOJoIcXRAkqiRRRHC1heXkVJtJDSVNAvjRZRWlBESUEhpQXFlEZHHxdRkl+oVhIRGWNm2cA/AO8CWoDfm9nP3f3l8z1XoEHG3X817ulO4MPz9bOzsrLG/nc4U/FEnN7BAXoG++gZ6KdnoI+ewX56B/vpHRygd7B/7Hlf6otj9Aukf02E44MD9A910T80mDw+NEg8MXET+0RysrOJ5uaRH8kjmptHXm6E/NwIebkRct7sI4+csS1CdmrLIS+1H30tN7VFyCGXrLHnOanH4/c5Y/s/btljm53xOGvca4alXrPUs7kfw+Q4idQWx3GcOAniOInUPvn8zMcjZ2zxscfDqcej+xhxRogzTIIYIwwTZ5g4Q6l9jBGGUvsYcYbG7YcYYaS+kKHhGIPDMYaGYwwMDzEYG2JgeGjSLpeJZGdlU5QXpSAvP7VFKc4voSIvysqKpRTm5VOYnwzVRfkFFOcXUJRfQFH+H5+Phpbi/EKK8qO6WaCIzLaLgf3u/iaAmf0AuBkIV5A5y23AD4Mu4nxkZ2VTWpD8X+hscHdiI8P0DQ3SHxtgMBajPzbIQGyQvqHkfmBo6IwvuoFYcht9HhsZZnA4xmBsKPnFuCaX3uEYHSPDDA4PMDQSIzY8zNDIMEPDMWLx4Rl9Wc6mLMvCzMgywywZbswYmyEyGnbGzxgZ7RZ0fOy5eyq0eCL52P0tq6XOt5zsbCLZueTlRsjLySWSm0teTjJoRnIi5OcWUZAbIVpaPhZAIzm5Y8G0IJJPfiRyRlAtyMsnGsmnIC8vuU+9pyASpTAvn0hOrmbXiEi6Ww4cGfe8BbhkJiea8yBjZr8Blpzj0Jfd/Wep93wZGAHuneQ8twO3p54Old521Z7ZrjVEKoH2oIuYLQlPgMM041Sorn0kHmckHp90UPp5CtX1zwFdv65/YV//bRMeaZrHKvgDxx5ZwV9Wvs3T5JvZc+Oeb3f37W/znG8x50HG3a+f7LiZ/RnwfuA6n2T0berit6c+85y7b53NOsMkk68/k68ddP26fl1/pl7/WYFgzrn7DXP8I44CNeOer0i9dt4CHX2XGrH8BeAmd+8PshYRERGZN78HVpvZKjOLAB8Dfj6TEwU9RubvgTzg16k+/Z3u/m+DLUlERETmUmq28l8Aj5Ccfn23u++dybmCnrXUOMOPznofW8hk8vVn8rWDrl/Xn9ky+foX3LW7+0PAQ2/3PDbVonAiIiIi6UorVImIiEhohTbImNk3zOwVM/uDme0ws0VB1zRfzOwjZrbXzBJmljEj+M3sBjN71cz2m9kXg65nPpnZ3WbWZmYZueyAmdWY2WNm9nLq7/5ngq5pvphZvpk9a2Yvpa79L4OuKQhmlm1mL5jZPwddy3wzs4NmttvMXpzv2UthENogQ/L2Bhvc/QLgNZK3N8gUe4APAo8HXch8Gbec9XuAZuAWM2sOtqp5dQ8w19Mh09kI8Hl3bwYuBf48g/78h4B3uvuFwCbgBjO7NOCagvAZYF/QRQToWnfflKnTzycT2iDj7r9y99EbxewkOQc9I7j7Pnd/Neg65tnYctbuHgNGl7POCO7+ONARdB1Bcfdj7r4r9biH5Bfa8mCrmh+e1Jt6mpvaMmpwo5mtAN4HfCfoWiT9hDbInOU24OGgi5A5da7lrDPii0zOZGZ1wGbgmWArmT+pbpUXgTbg1+6eMdee8k2Sa44Fe8+R4DjwKzN7PrXKvYwT9Doyk5qt2xuE0XSuXSTTmFkRcD9wh7t3B13PfHH3OLApNRZwh5ltcPeMGC9lZu8H2tz9eTO7Juh6AnKlux81syqS6669kmqlFdI8yMzW7Q3CaKprz0Cztpy1hJOZ5ZIMMfe6+wNB1xMEd+8ys8dIjpfKiCADXAHcZGbvBfKBEjP7vrt/IuC65o27H03t28xsB8mudgWZlNB2Len2Bhln1pazlvCx5NLf3wX2ufvfBV3PfDKzxaOzMs0sCrwLeCXYquaPu9/p7ivcvY7k7/2/ZFKIMbNCMysefQy8m8wJsdMS2iBD8vYGxSSb2V40s/8ddEHzxcw+YGYtwGXAg2b2SNA1zbXUwO7R5az3AT+a6XLWYWRm9wFPA01m1mJmnw66pnl2BfBJ4J2p3/cXU/9DzwRLgcfM7A8kA/2v3T3jpiBnsGrgCTN7CXgWeNDdfxlwTWlFK/uKiIhIaIW5RUZEREQynIKMiIiIhJaCjIiIiISWgoyIiIiEloKMiIiIhJaCjIiIiISWgoyIiIiEloKMiJwXM7vbzNrMTKuLikjgFGRE5HzdQ/JePyIigVOQEZFzMrPHzOxdqcd/Y2b/CyB1192OQIsTEUlJ67tfi0igvgb8lZlVAZuBmwKuR0TkLRRkROSc3P3x1F2nPwdc4+7xoGsSETmbupZE5JzMbCPJOy/H3L0n6HpERM5FQUZE3sLMlgL3AjcDvWamwb0ikpYUZETkDGZWADwAfN7d9wF/TXK8zOjx+4CngSYzazGzTwdTqYgImLsHXYOIiIjIjKhFRkREREJLQUZERERCS0FGREREQktBRkREREJLQUZERERCS0FGREREQktBRkREREJLQUZERERC6/8DXyfyw9/p90kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "w = paddle.to_tensor([0.2, 2])\n",
    "model = OptimizedFunction(w)\n",
    "opt = Adagrad(init_lr=0.5, model=model, epsilon=1e-7)\n",
    "train_and_plot_f(model, opt, epoch=50, fig_name='opti-vis-para2.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，AdaGrad算法在前几个回合更新时参数更新幅度较大，随着回合数增加，学习率逐渐缩小，参数更新幅度逐渐缩小。在AdaGrad算法中，如果某个参数的偏导数累积比较大，其学习率相对较小。相反，如果其偏导数累积较小，其学习率相对较大。但整体随着迭代次数的增加，学习率逐渐缩小。该算法的缺点是在经过一定次数的迭代依然没有找到最优点时，由于这时的学习率已经非常小，很难再继续找到最优点。\n",
    "\n",
    "**简单拟合实验**  训练单层线性网络，验证损失是否收敛。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAEICAYAAAB74HFBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xl8XHW5+PHPM2v2Jm3StOneUgotSwstqyKrIqK4X3FD5YpXvSrqVVyuV9xx/+l1RUG8LigCKvsiIMhWKNC9QPclTbM0+zLreX5/zMlkJpk0STPtTDLP+/XKqzPnnDnnSZXTZ77f5zxfUVWMMcYYY8zh8eQ6AGOMMcaYicySKWOMMcaYcbBkyhhjjDFmHCyZMsYYY4wZB0umjDHGGGPGwZIpY4wxxphxsGRqkhGRTSJybg6vP1dEukXEewTOrSJyTLbPa4wxg4nIfPee4xvl8TeJyNePdFwmP1kyNcmo6jJV/SeAiFwrIr8/ktcTkV0icmHK9feoapmqxo/kdY0xxph8YcmUGdZov5EZY4wxhcySqUmmf6RIRC4GvgD8mzvtts7dP0VEbhCRBhGpF5Gv90/Jicj7ROQJEfmhiBwErhWRRSLysIgcFJEWEfmDiFS6x/8OmAvc6V7js4OHxkWkTkTuEJFWEdkmIh9MifVaEblFRP5PRLrcKcqVo/w9p7ifaxaR3SLy3yLicfcdIyKPikiHG/Of3e3i/m5NItIpIhtE5ISs/eUbY44Y915ym/vf/E4R+XjKvmtF5FYR+bN7L3leRE5O2X+8iPxTRNrd+8wbUvYVi8j33ftIh4g8LiLFKZd+l4jsce8lXxxDvB9073mt7j2wzt0+7H1IRC4Rkc3u71AvIv81rr80c9RYMjVJqep9wDeBP7vTbv03lpuAGHAMsAJ4NfDvKR89HdgB1ALfAAT4FlAHHA/MAa51r/EeYA/wevca38kQyp+Afe7n3wp8U0TOT9n/BveYSuAO4Cej/BX/F5gCLAReBbwXeL+772vAA0AVMNs9Fvd3PQc41v3s24GDo7yeMSZH3C9KdwLrgFnABcDVIvKalMMuA/4CTAX+CPxNRPwi4nc/+wAwHfgY8AcRWeJ+7nvAqcBZ7mc/Czgp530FsMS95v+IyPGjiPd8EvfNtwMzgd0k7nNw6PvQDcCHVLUcOAF4eKRrmfxgyVQBEZFa4BLgalXtUdUm4IfAO1IO26+q/6uqMVXtU9VtqvqgqoZVtRn4AYnkZTTXmwOcDVyjqiFVXQv8mkTi0+9xVb3HrbH6HXByhlMNPq/XjfnzqtqlqruA7wPvcQ+JAvOAOve6j6dsLweOA0RVt6hqw2h+F2NMTq0CalT1q6oaUdUdwK9Iv3c9p6q3qmqUxH2qCDjD/SkDrnM/+zBwF3C5m6R9APiEqtaralxVn1TVcMp5v+LeC9eRSOZGvEcB7wJuVNXn3XN9HjhTROZz6PtQFFgqIhWq2qaqz4/5b8rkhCVThWUe4Aca3OHuduCXJL6t9dub+gERqRWRP7lDzp3A74HqUV6vDmhV1a6UbbtJfLPsdyDldS9QNIparWr399g9zHk/S2JE7Rl3SP8DAO5N9CfAT4EmEbleRCpG+bsYY3JnHlDXf99y711fIDGC3i9571JVh4ER8Tpgr7utX//9oppE0rX9ENcefI8qG0W8daTcn1S1m8To06wR7kNvIfGFd7dbqnDmKK5l8oAlU5ObDnq/FwgD1apa6f5UqOqyQ3zmm+62E1W1Ang3iURluONT7Qemikh5yra5QP1YfokMWhgYfRpyXlU9oKofVNU64EPAz8RtqaCqP1bVU4GlJIbZPzPOWIwxR95eYGfKfatSVctV9ZKUY+b0v3BHnGaTuAftB+b011S6+u8XLUAIWJTlePeTcn8SkVJgGgP3qIz3IVV9VlUvI/EF92/ALVmOyxwhlkxNbo3A/P6biDuU/ADwfRGpEBGPJArMDzVtVw50Ax0iMouhyUcjibqlIVR1L/Ak8C0RKRKRk4ArSYxuHTZ3SvAW4BsiUi4i84BP9Z9XRN4mIrPdw9tIJHyOiKwSkdPdGooeEjdRZ+gVjDF55hmgS0SucQvGvSJygoisSjnmVBF5szuyfTWJL45PA6tJjCh91q2hOhd4PfAnd7TqRuAHboG7V0TOFJHgOOO9GXi/iCx3z/VNYLWq7hruPiQiARF5l4hMcacqO7H704RhydTk9hf3z4Mi0j/3/l4gAGwmkWjcSqJAcjhfAU4BOoC7gdsH7f8W8N/u0HumJ08uB+aT+Kb2V+DLqvqPsf8qQ3yMxI1oB/A4iYLTG919q4DVItJNoqj9E26NRQWJOos2EkPwB4HvZiEWY8wR5H6BuhRYDuwkMaL0axIF3P3+Dvwbif++3wO8WVWjqhohkTy91v3cz4D3quqL7uf+C9gAPAu0At9mnP82uve4LwG3AQ0kRr7667sOdR96D7DLLan4DxK1V2YCENVDzdIYY4wx+U1ErgWOUdV35zoWU5hsZMoYY4wxZhwsmTLGGGOMGQeb5jPGGGOMGQcbmTLGGGOMGYejupBtdXW1zp8//2he0hiTY88991yLqtbkOo7xsvuXMYVntPevo5pMzZ8/nzVr1hzNSxpjckxEdo98VP6z+5cxhWe09y+b5jPGGGOMGQdLpowxxhhjxsGSKWOMMcaYcRgxmXLXVHtGRNaJyCYR+Yq7/SYR2Skia92f5Uc+XGOMMcaY/DKaAvQwcL6qdrsLMz4uIve6+z6jqrceufCMMcYYY/LbiMmUJrp6drtv/e6Pdfo0xhhjjGGUNVMi4hWRtUAT8KCqrnZ3fUNE1ovID0UkOMxnrxKRNSKyprm5OUthG2OMMcbkh1ElU6oaV9XlwGzgNBE5Afg8cBywCpgKXDPMZ69X1ZWqurKmZnR9+2LbDtL5pYeJ7Wkf1fHGGJMv1FHCG/cR3dua61CMMUfJmJ7mU9V24BHgYlVt0IQw8BvgtGwFFdvRRtfXHyO+rzNbpzTGmKNCPEJkSwPRlw/kOhRjzFEymqf5akSk0n1dDFwEvCgiM91tArwR2Ji1qEQSfzpWmmWMmXh8dZXEDrSjdg8zpiCMZmRqJvCIiKwHniVRM3UX8AcR2QBsAKqBr2cvKjeZsvuQMWYcRGSOiDwiIpvd1i6fcLd/V0RedGs+/9r/hTFbfHWVEIkTP9iVzdMaY/LUaJ7mWw+syLD9/CMSEQMDUzYyZYwZpxjwaVV9XkTKgedE5EHgQeDzqhoTkW+TqAHNWPd5OLwzE7lZfH87vpqKbJ3WGJOn8rMDenJkypIpY8zhc2s7n3dfdwFbgFmq+oCqxtzDnibxcE3WeIr8eKaWEttvD9EYUwjyM5mymiljTJaJyHwSo+yrB+36AHDv4OPdzxx2axdfXRXxpk40Gj+MaI0xE0l+JlPuyJQNTBljskFEyoDbgKtVtTNl+xdJTAX+IdPnDqe1Sz9fXSU4SrzJnko2ZrLLz2TKaqaMMVniLoN1G/AHVb09Zfv7gEuBd7krPWSVt7YCPEJsf1u2T22MyTOjWZvv6LOaKWNMFritW24AtqjqD1K2Xwx8FniVqvYekWv7vHinV1jdlDEFIE9HpqxmyhiTFWcD7wHOF5G17s8lwE+AcuBBd9svjsTFfXWVOK09OKHIkTi9MSZP5OXIlFifKWNMFqjq4wwUDqS652hc31dXRfj53cT3d+BZOLaaK2PMxJGnI1PunzYyZYyZwDzTyiDgJdZgU33GTGb5mUwln+azZMoYM3GJR/DNqCS2v83uZ8ZMYvmZTFnNlDFmkvDVVaLdYbQrlOtQjDFHSH4mU1YzZYyZJLx1iaVl7Kk+Yyav/EymrGbKGDNJeCqKkdKg1U0ZM4nlZTIl1mfKGDNJiAi+ukriDe2ofUE0ZlLKy2TKaqaMMZOJb2YlGo7htHbnOhRjzBGQn8mU1UwZYyYRq5syZnLLz2SqP5eykSljzCTgKQ7gqSqxuiljJqn8TKasZsoYM8n46qqIN3agsXiuQzHGZFl+JlNWM2WMmWS8MyshrsSbOnMdijEmy/IymbK1+Ywxk41vxhQQsbopYyahvEymrM+UMWayEb8X7/Ryq5syZhLKz2TKaqaMMZOQr64Sp6UbDUdzHYoxJotGTKZEpEhEnhGRdSKySUS+4m5fICKrRWSbiPxZRAJZi8qtmbKn+Ywxk4l3ZhUAsYaOHEdijMmm0YxMhYHzVfVkYDlwsYicAXwb+KGqHgO0AVdmLyqrmTLGTD7emjLwe4ntb8t1KMaYLBoxmdKE/ra9fvdHgfOBW93tvwXemLWorGbKGJMFIjJHRB4Rkc3uyPon3O1TReRBEdnq/ll1VOLxePDNmELcitCNmVRGVTMlIl4RWQs0AQ8C24F2VY25h+wDZg3z2atEZI2IrGlubh5VULY2nzEmS2LAp1V1KXAG8FERWQp8DnhIVRcDD7nvjwpvXSVOVwinK3S0LmmMOcJGlUypalxVlwOzgdOA40Z7AVW9XlVXqurKmpqa0X3I+kwZY7JAVRtU9Xn3dRewhcQXv8tIjKhDtkfWR+DrX1rGnuozZtIY09N8qtoOPAKcCVSKiM/dNRuoz15UVjNljMkuEZkPrABWA7Wq2uDuOgDUDvOZMY+sj8QzpQQpDRLd1ZKV8xljcm80T/PViEil+7oYuIjEt7tHgLe6h10B/D1rUdnafMaYLBKRMuA24GpVTWtBrqrKMF/dDmtkfeRY8B8znfj+NpyecFbOaYzJrdGMTM0EHhGR9cCzwIOqehdwDfApEdkGTANuyFpUYjVTxpjsEBE/iUTqD6p6u7u5UURmuvtnkqgHPWoCx9SCQnT7Ub2sMeYI8Y10gKquJzE0Pnj7DhL1U9nnsZopY8z4iYiQ+KK3RVV/kLLrDhIj6teR7ZH1UfBUFOOtrSC6tZHAibOR/i+QxpgJKT87oPffVyyXMsaMz9nAe4DzRWSt+3MJiSTqIhHZClzovj+q/Itn4HT22cLHxkwCI45M5YK1RjDGZIOqPs7A17PBLjiasQzmn19N6OltRLc24qudkstQjDHjlKcjUzbNZ4yZ3MTvxb+ghujOFjQaz3U4xphxyM9kylojGGMKgP+YWojFrU2CMRNcfiZT1hrBGFMAvLUVeMqLiG5rzHUoxphxyM9kymqmjDEFQETwL64lfqADp6sv1+EYYw5TfiZTVjNljCkQ/kWJ5uuRrTY6ZcxElZfJlFjNlDGmQHjKgnjrKolua0JtNN6YCSkvk6nkg8w2MmWMKQCBxTPQnjBxW/zYmAkpP5Mpq5kyxhQQ39xpEPDZVJ8xE1R+JlNuzZQ9zWeMKQTi8+BfWENs90E0HMt1OMaYMcrPZMpqpowxBSawuBbiDtGdzbkOxRgzRvmZTFnNlDGmwHimleGpLLGeU8ZMQHmaTFnNlDGmsCR7TjV3EW/vzXU4xpgxyMtkSqzPlDGmAPkXTQcRolaIbsyEkpfJFJCom7JcyhhTQDzFAXxzphLd3mgP4BgzgeRvMiX2NJ8xpvD4j6lF+6LE9rXmOhRjzCjlbzLlEauZMsYUHN+cKqQkQGRTfa5DMcaMUv4mUyJWM2WMGTcRuVFEmkRkY8q25SLytIisFZE1InJaLmNMJR4PgRNmEz/QQayxI9fhGGNGIX+TKY8QfmA7kef25zoSY8zEdhNw8aBt3wG+oqrLgf9x3+eNwJIZSJGf8Lq9uQ7FGDMKIyZTIjJHRB4Rkc0isklEPuFuv1ZE6t1vdmtF5JJsBiYC0RcO0Lzy+mye1hhTYFT1MWBwAZICFe7rKUBefWsTn5fAslnE69uIN3flOhxjzAh8ozgmBnxaVZ8XkXLgORF50N33Q1X93hGJrL8LujHGZN/VwP0i8j0SXyrPynSQiFwFXAUwd+7coxcdEDhuJuEN+wiv30vJBUuP6rWNMWMz4siUqjao6vPu6y5gCzDrSAeWbNxpjDHZ92Hgk6o6B/gkcEOmg1T1elVdqaora2pqjmqAEvARXFpHbM9B4q09R/XaxpixGVPNlIjMB1YAq91N/yki690Cz6phPnOVW+C5prl5DGtOpYxM1cu1tF5+61hCNcaYQ7kCuN19/RcgbwrQUwWW1oHfS3j9nlyHYow5hFEnUyJSBtwGXK2qncDPgUXAcqAB+H6mzx32N7tBA1N9f9qY+ThjjBm7/cCr3NfnA1tzGMuwJOgncHwdsZ0ttsSMMXlsVMmUiPhJJFJ/UNXbAVS1UVXjquoAvyLb3+wy1Ez13bYZgHh9J+o4Wb2cMWZyEpGbgaeAJSKyT0SuBD4IfF9E1gHfxK2LykeBZXXg9RBZb0/2GZOvRvM0n5CoJ9iiqj9I2T4z5bA3AdkdOspQM9X61lvQUJQDs39A2xV/y+rljDGTk6perqozVdWvqrNV9QZVfVxVT1XVk1X1dFV9LtdxDsdTFCBw3EyiO5pwuvpyHY4xJoPRjEydDbwHOH9QG4TviMgGEVkPnEeiiDNrZJin+bQvBkDf79dn83LGGJO3AifMSvTeW78v16EYYzIYsTWCqj7OkAomAO7Jfjgj07hN7xljCounJIh/8QyiLx8gePJcPGXBXIdkjEmRtx3Qh02aYpZMGWMKT/DE2aAQ2WijU8bkm7xNpogPsy7fcNuNMWYS85QV4T9mOpGXD+D0RnIdjjEmRf4mU8OMQNk0nzGmUAVPmgOOQ2RTfa5DMcakyNtkSoebzrNpPmNMgfJUFONfUEPkxf04oWiuwzHGuPI2mRo2abJpPmNMAQucPAdijvWdMiaP5G8y5WROmoYdsTLGmALgrSzFf+wMIpv3W1d0Y/JE/iZTw9CucK5DMMaYnAqeMg/8HkKrt6Nqo/XG5NrES6Z6B+oE7CZijClEnuIAwRXziO9vJ7bnYK7DMabgTehkaripQGOMmewCx9XhqSwh9MwONBbPdTjGFDRLpowxZgISj1B0xiK0O2yNPI3JsYmdTFnPKWNMAfPNrMQ3v5rw+n043aFch2NMwZrQyZTayJQxpsAVrVoIAqFnduQ6FGMK1oRLpkL3bht4Y8mUMabAecqCBE+aQ2z3QWL723IdjjEFaeIlU3e8NPDGGngaYwyBZbOR8iJCT29HHSt/MOZom3DJVBobmTLGjEBEbhSRJhHZOGj7x0TkRRHZJCLfyVV82SA+D0WnLcTp6COypSHX4RhTcCZ0MmU1U8aYUbgJuDh1g4icB1wGnKyqy4Dv5SCurPLNmYpvVhXhF3bj9EVyHY4xBWVCJ1P2NJ8xZiSq+hjQOmjzh4HrVDXsHtN01APLMhEhePpCiDuE1+zKdTjGFJSJnUzZyJQx5vAcC7xSRFaLyKMisirTQSJylYisEZE1zc3NRznEsfNOKSGwdBbRbY3EGjtzHY4xBcOSKWNMIfIBU4EzgM8At4iIDD5IVa9X1ZWqurKmpuZox3hYgsvnIKVBQo+/jEatM7oxR4MlU8aYQrQPuF0TngEcoDrHMWWF+H0Uv/JYnM4+Qmt25jocYwpC3iZTgbPmjHiMZmiNoOEYoQe3H4mQjDGTx9+A8wBE5FggALTkNKIs8s2sJLBsFtEXG4juG1wuZozJthGTKRGZIyKPiMhm9xHiT7jbp4rIgyKy1f2zKpuB1TxxJSVXnHzogzKMTHV9+3EOvvp3hB6ybsDGGBCRm4GngCUisk9ErgRuBBa67RL+BFyhqpNqqDt4yvzEQsiPv4wTio78AWPMYRvNyFQM+LSqLiVRX/BREVkKfA54SFUXAw+577NKwynz/UW+oQdkSKac9sT6VNHn9mc7HGPMBKSql6vqTFX1q+psVb1BVSOq+m5VPUFVT1HVh3MdZ7aJz0PxOUsSo/VPbWOS5YrG5JURkylVbVDV593XXcAWYBaJHi2/dQ/7LfDGbAen4VjydfV97x66P+6gqvTd9RLRDY0A+OZXAhDbYcsqGGMKm3daGcEV84jtaiG6I/+fRjRmohpTzZSIzAdWAKuBWlXtb7V7AKgd5jOH/WhxxZfPHThP0Dv0AEcJ3fkSra+/maaTfk7vH9cj7ghWfHfHmK5ljDGTUeCE2XinVxB6ehtOdyjX4RgzKY06mRKRMuA24GpVTWtg4tYaZBxDHs+jxf6TZ+A7dlrijTdDqI7iNPYk30ZW16Mxt5FnzBp6GmOMeITic44FB/r+9bJN9xlzBIwqmRIRP4lE6g+qeru7uVFEZrr7ZwJHpoOw12394sucTJHSGUb7ogOLHw/pGGOMMYXJU15M0ekLiR/oILLZ6kmNybbRPM0nwA3AFlX9QcquO4Ar3NdXAH/PfniAJ5EViTdDdhTX5H4A7YkOjEgN7b9njDEFy7+4Ft+cqYSf20m8rWfkDxhjRm00I1NnA+8BzheRte7PJcB1wEUishW40H2ffe70XqZFjZuW/4Le365Lvtfe6MA0n+VSxhiTJCIUnb0Y8fvoe+wl1NY2NSZrMvQbSKeqjzN8anJBdsMZKjkilaFBJ0Dksd3J19pnI1PGGDMcT3GAorOOoe/hLYSf3UnRGYtyHZIxk0LedkBP6i88H8XSMTYyZYwxh+afV01gaR2RLfuJbGvMdTjGTAr5n0z110SNYkhae6P2FJ8xxowguGoh3hlTCD25jXhLV67DMWbCy/tkquiihQB4ppeOeKz2xQZGpmwRZGOMyUg8QvF5xyFFfnof3oITiuQ6JGMmtLxPpsq/ch612z6Ob+HUkQ+OOwMjWMPUWBljjAFPUYCS849HQxH6Hnkx40M+xpjRyftkSrwefIsSiZRMCR7yWI05yWk+e1LFGGMOzVtdTtFZi4kf6CC8ZmeuwzFmwhrxab58UrP6g4TueAnfwipa33rLkP1Ocy/d338q8cZGpowxZkSBY2pxWrqJbKrHM62MwKLpuQ7JmAkn70emUvmXVFP+mbOHrZ/SzvDAm2FGpmK72oi80JBxnzHGFKLgaQvw1lYQemIr8YPduQ7HmAlnQiVTSZm6oQ82zPx/44If0XzKL7MckDHGTFzi8VB83vFI0Efvw5txQtFch2TMhDIhkynxjJxM6QjTfOpYTZUxhUBEbhSRJhHZmGHfp0VERaQ6F7HlE09xgJLzl6J9Efr+ucUK0o0ZgwmZTDGKZGqkvlROk61NZUyBuAm4ePBGEZkDvBrYc7QDylfemnKKzlxMvKGD0FPbULWEypjRmMTJVOImoJEYkWf2Ebr75fTdezuPRGTGmDyjqo8BrRl2/RD4LGAZQ4rA4loCJ80h+vIBwi/sHvkDxpgJmkx5Rw67vzVC23v+SvPpv+bgpX9EVfFUlwAQr7dkyphCJSKXAfWqum7EgwtQ8JR5+BfXElm3l8iW/bkOx5i8N6FaI/QbTc1UfwF6362bk5u0KwxBb+J1X+yIxGaMyW8iUgJ8gcQU30jHXgVcBTB37twjHFn+EBGKzlqMhqKEnt6OFPvxz6/JdVjG5K2JOTI1hmm+1AWPnbbQwIC+NfU0plAtAhYA60RkFzAbeF5EZgw+UFWvV9WVqrqypqawkgnxCMXnHod3egV9j75ErKE91yEZk7cmZjI1itYIyQ7oKYmX09aXHLEa6Wk/Y8zkpKobVHW6qs5X1fnAPuAUVT2Q49Dyjvi8lFy4FE9FMb0PbSbeaj2ojMlkYiZTYxmZSjlWW/sG9sdsZMqYQiAiNwNPAUtEZJ+IXJnrmCYSCfopuegExO+l94GNOF2hXIdkTN6ZPMnU4E0xB6c7nFZflZjmc5Msm+YzpiCo6uWqOlNV/ao6W1VvGLR/vqq25Cq+icBTFqTk1SegcU0kVKFIrkMyJq9MyGQqUwF6zTMfTHsf39NBQ/m30grNnVab5jPGmMPhrSql5MJlOD1heh/chEbtIR5j+k3IZCpTawSpCI74Mac9ZXh6mGm+6JZmnPa+jPuMMaaQ+WorKD73OJyD3fQ+sAmNWEJlDEzUZCrDyJSnfORkSrsjIz7N17T0pzSf8evxRGeMMZOWf+40is89nnhzF70PbLSEyhgmUTI1mpGprq8+itPSCxx6mi/20sHDj80YYyY5//xqis87jnhLNz33b0TDllCZwjZiMpVpkVARuVZE6kVkrftzyZENc1BMmZKpEv/YTmJP8xljzGHzz6um+PzjcVq76XlgAxqO5jokY3JmNCNTN5FhkVDgh6q63P25J7thHZqGhn4LEhlFu4RUKdN88ZYenK4w6liCZYwxo+WfO81NqHrcESpLqExhGjGZOsQioTnjnVWOd94Upvz08AfE+qf5Oj77AAdqvkvjwh9B1JIpY4wZC/+caRSfvxSnvYee+zbghCyhMoVnPDVT/yki691pwKrhDhKRq0RkjYisaW5uHsflUs5Z5GfGrk9S9pHTMu4v+6+zRj6JO83X/d0nAXBaetGUqT+rATDGmNHxz5lKyQXLcDp66b1vg/WhMgXncJOpn5NY32o50AB8f7gDj/TaVoHz5uOZXgpA8NWLKHnf8lEtN5Pxab5oPPlSU14bY4w5NN+sqkRC1dlH770bcHotoTKF47CSKVVtVNW4qjrAr4DMQ0RHQc3D72Nm42cAqL7/PVT95o0Z+1ANlulpPk2d5rMCdWOMGRPfrKpEY8/uED13ryXe0ZvrkIw5Kg4rmRKRmSlv3wRsHO7YXJDRjExlSpZSR6YsmTLGmDHz1VVS+tqTIObQe/c6Yk2duQ7JmCNuNK0RMi0S+h0R2SAi64HzgE8e4TjHZlQjU0OTJaczPPAmQzKlfVE6rnkQp8eGr40xZjje6nJKX3cyEvDRe98Gonusd5+Z3HwjHaCql2fYfEOGbfljVDVTQ6f5nAPdA28yJFM9NzxP93eeQPweKr5+wXgiNMaYSc1TUUzJ606m9x+b6Ht4M3rGMQSOmznyB42ZgCZmB/QRZGrqOVjPj1fTuOR/07Y5zQPz+5mm+cTvBSDe2DPOCI0xZvLzFAcovfgkfLOqCD21jdDzu1C1RebN5DMpk6lM03wV33v1kG2xl9OHnlvf/peUnRmSqcoiYNCCycYYY4Ylfi/FFyzDv7iWyLq9hJ7Yag2SzaQzSZOp9JEpT00J3pqSMZ2if2TK6Q7Tdd2/iLf0IIHEyJS29WUnTmOMKQDiEYrOXkxg+VyiWxvp+8dmWyDZTCqTM5kaRCNxpHzkhZDTxBziTd1KFzRrAAAgAElEQVQ0lH+Lzs8/RPsH/p7sPeW02ciUMRPFMOuLfldEXnQbD/9VRCpzGWMhEBGKVsyj6KxjiO1vo+eutcQ77IupmRwmZTKlg6fhog5SFhjbOWIOTsNAQXq8viu53IxN8xkzodzE0PVFHwROUNWTgJeBzx/toApVYMlMSl59IhqK0nPXWmL78mq1MmMOy6RMppyD6d92NBJHgt6xnSTmQEohu9MVRiNuH6pM3dONMXkp0/qiqvqAqvbPMz0NzD7qgRUwX10lpa9fgac0SO8/NhHesM8K082ENjmTqdZBQ8cxB/wDyZRnVvnIJ4k5kFJ6pV2RgaaekvlpwejGRnp+9dxYwzXG5NYHgHsz7TgSa4uaBE95EaWvOxnfvGrCa3bS99hLaMyW8TITU2EkU5AsHvevmIFnStGI59C4pi983B1JX24mg6aV19N+1Z32pIoxE4SIfBGIAX/ItP9Iry1a6MTvpfjc4wieMo/YjmZ67l6H021lFGbimZTJVNk1ZyOlfqr++BYIeqn89RvwnzCdwDnzqPz5paObpos5EElZXqY7dWRqmM+EE/uH1GwZY/KOiLwPuBR4l9ocU86ICMGT51J84TKcrhA9d64ldqAj12EZMyYjdkCfiIouXERd9xcBKLn8xOT2mkffD4CGRx5K1piTLDjv57S4TT2HmeaTsgDaHSHe3Itn6thaMRhjjh4RuRj4LPAqVbXVePOAf85UPJcup++hzfTet4HgqfMJnDALGeZ+a0w+mZQjUyPRyMjJ1MEL/4/OLzyUtq3rG/9KvHD/29a4Q+gf2+n49P2o4yBTEu0XnGbrkG5MvhhmfdGfAOXAgyKyVkR+kdMgDQDeyhJKX78c39ypiTqqBzfhhGwtVJP/JuXI1IgGJVO+46qJ7+9CUxc6BiKP7T7kabq++ihdX30UgJIrV+CpCOLUdw2MYBljcm5Cri9awCTgo/i844m+2EDomR30/P0Fil+1BN8MawVm8ldhjkyF0zvvVnztPKTEP/oTuIXpoQe2D5yzM4y4he2pa/wZY4wZGxEhcHwdpZcuR3xeeu/bQHjtbtSx0jaTnwozmepKHzbWmJN82m9Un3eTqdQFlZ22EFKcGOhzBo1w9ev82qP0/OaFsYZrjDEFyTutjNI3LMe/oIbwC3vovX8DTq9N+5n8U5DJ1GAajsMYkqnkIsipyVRqnVQ0c01W1/88QvsH/n44IRpjTEESv4+ic5ZQ9IrFxJu76Pn788Tq23IdljFpLJkCCMcGRqZG8eBIst9USjIVb+oBHbTfGGPMuIkIgcUzKH3DcqTIT+8DGwk9vd2afJq8YckU6cvNTL317SN/oH9kypsyMtXUM9C/apiRKWOMMYfPW1lK6euXEzi+jsiW/fT8/QViTZ25DssYS6YAAmfNIXjeAgA81aPoDxWNJ1ohpIxMdX/3SSL/2gMMjEyFn9hDvVzLgbk/yH7QxhhTgMTnpeiMRZRcfCLqOPTes47Qmp2orZlqcqggWyP4TphObGMTANPX/gf+k2fgP6mWkg+swDO1eMTPa1+M/d6v4p1TkfkAd2Sq5RU3AhDfm/7NSaNxxD/GhZeNMcYk+WZWUnbZKYSe2UFkwz5i+1opfuUSvNPKch2aKUAFOTI1/fkPUdf3RWbptfhPngEkvu34l00H39C/kopvX5jxPIOTpH4jNQV1Dg5tneB0hGiY/h3Cj+0aIXpjjDHg9qR6xbEUX7gMDUXpuXMt4bV7rIWCOepGTKZE5EYRaRKRjSnbporIgyKy1f2z6siGmV3i9yJFmftKiXdoBXrZp84c2wUyFKBrZKC3ldMxtHVCdO0BnOZeOr/48NiuZYwxBc4/ZyqlbzwV34Jqwi/spufutcRbbSUKc/SMZmTqJuDiQds+BzykqouBh9z3k0OGkSm8YxvA0wwF6PGG7oE3GfZLaSK5067MPaqMMcYMz1Pkp+RVx1F87nFod5ieO54n9OxOe+LPHBUjZgmq+hjQOmjzZcBv3de/Bd6Y5bhyJ0MyNdaFNjO1Rojv68y4P/L0XpzOELjD0k63NaQzxpjD5V9QQ9mbT8V/TC2Rjfvo/qv1pTJH3uHWTNWqaoP7+gBQO9yBInKViKwRkTXNzc2HebmjRzKNTKUo/9p5I58kw8hTfzE6kFwbsPHEn9F85g0cvOxPicahDO3ObowxZmwk6Kf4FcdS8toTEa/Q+8BGeh99EafP7q/myBh3AbqqKsl2lRn3X6+qK1V1ZU1NzXgvd+SNkEx5KotGPEV0YxOhf2wfdr9G4mgsnnyiMPLPXcmi9eGm+aIbGmm76g57/NcYY0bJN6OS0stOIbB8LrFdLXTf/hyRlw+Q+GfLmOw53NYIjSIyU1UbRGQm0JTNoHJJPOnJ1IzmzwAw9W/vwGnpHWjYeQixDU0cvOh3w+7XSDw5EpXkLr6sfbEMn4DWt/+F2IstlH3qTPzHTYCk1Bhj8oB4PRStmId/QQ2hJ7cSemIr0W2NFJ2xCO9Ua6NgsuNwR6buAK5wX18BTKoF54ouPZbAWXPwzpuCt7oUgOLLjqP0ylMyjlyNqmt6qkgcDaUnTUOSq0H6F1FWq6kyxpgx81aWUPLakyg6ezFOey89d7xA31PbcELRXIdmJoERR6ZE5GbgXKBaRPYBXwauA24RkSuB3cAYs4n8Nu3Odw67T/xDkynPzLF9u9FoHIYkU0NHpDQaJ3TfNoouPRYpCwCZ2yoYY4wZmYgQOHYG/nnTCL+wh8iL+4ntbCa4Yh7+JTPTVrUwZixGTKZU9fJhdl2Q5VgmhgwjU56qkbump8kwMtX+H3elvY+uP0DTyb8AYNo97xpIpjI0/AQSTwQqeKaMXNNlTCERkRuBS4EmVT3B3TYV+DMwH9gFvF1V7ZGvAiFBP0VnLMK/ZAah1dsJPb2dyEsNFJ2+CN/MylyHZyagguyAPi4Zvrl4qgYSmMBZc0Y8hWaa5hs04tTz6+eTr2PbW/GUBwFwDvZlPGfD9O/SUHndiNc2pgDdRCH1yjOj5q0qpeQ1J1J83vFoNE7vfRvofWQLTnco16GZCaYg1+YbD6c1PZkp/Y+VaSNTUh4Y8Rxt77qdojcdd+iDUhqFakc42dTTack8MsUINVfGFCpVfUxE5g/afBmJ8gVI9Mr7J3DNUQvK5A0RwT+/Gt/sKiIb9xFev4/uva0EltYRPHEOErR/Js3IbGRqjJzG9CUKKn9+KRL0UXbN2VQ/eWXGbumBM2cP2Rb664uHvE7q3L3TEUo2n3DaMo9MGWPGZFS98iZanzxz+MTnJbh8XqLh5/xqIhv20X3bs4Q37rOWNGZElkyNUdFlSwbepKzjN+W6iwieOSfZ4qBf2afPxDNjjAXqjpM2nei09hHf05F4445ARTc0Uu/9CvVy7dDPZuB0huyGYEwGh+qVN+H65Jlx85QVUXzOEkrfsALPtDLCz+6k+/Y1RLc3WX8qMywbvxyjwCl1zNJrie1uR0qGLpY8+Gk77YkiRWP8a446aYla7w0vDJzPTdaaTvr5wLaU/8C1PYRMLUmPIRyjYcp1lH7sNCp/fMnYYjFmcpq0vfJMdninlVH6mhOJ7W8j9OxO+h57Cc/GfRStXIBvVlWuwzN5xkamDpNvXiXemtIh2+MNXWnvnZ5IMpkKXrhwVOeO13cOu7hypn5UqU/4xTPUVGlvoo9Kz8/XjOr6xhSASd0rz2SPr66K0jesoPicJWgkRu8DG+m5fwOxps6RP2wKhiVTWeY0dKe9L//iOclkqujSY0d1jsZFPyb01y2Zd2boR5W6iHLq035OWx8ajg08OTiK7u3GTDZur7yngCUiss/tj3cdcJGIbAUudN8bk5GI4F80nbI3ryR42kKcg9303r2O3gc3Em/pGvkEZtKzZCrbnIEpt8A58/Avqab8f15F0ZuPp+R9y4ccPvXv78h4mtiWlozbB7dUGHys9iQ6pHf94Ekapn6bzq8+mvEzaSH3ROj5xbNWD2AmJVW9XFVnqqpfVWer6g2qelBVL1DVxap6oaq25jpOk//E6yG4bBZlbzuN4KnziTd30XPnWnr/sYn4we6RT2AmLUumsqz0o6uSr71u4bl3RjnTbvu3jA01A2cMfdLvUDJN87W987aB/W7i1PnpBwCIbWpC+w69XELX/zxC+4fvJnTHS2OKxRhjCpH4vQRPmkPZW1cRXDGPWGMnPXe8QO/Dm4m39Yx8AjPpWAF6llX+5HVM+d9L6Pnf1ZS89+QRj0/tYSJlgRHX3su07EyaQaNQ8fqukUem2hMN6pwD9s3KGGNGSwI+gsvnEji+jvDmeiKb6ontPohvfjXBk+bgnWYLKRcKG5k6AkSEso+fgady5GVm+ptxAlT99o0jHh95dPeQdgiptC+WNl0Xr+/MmExF1tTT9v6/EW/pGViqpidzIud0h4k3WqJljDGZSNBH0Yp5lL9tFYGT5hCrb0uMVD24kVhjR67DM0eBJVM5NH3TRxCfl+J3nZjYEPAOOWbaXcMvupyJhmIQGZgKdBp76Pzsg2nHRJ7eS/OqX9F701rC/9iRTOi0J/N0YPPK6zkw43tjisMYYwqNBP0UnTqf8redRvCUecSbu+i9Zz0996wjVt9mdamTmCVTOSRu8lT1uzczS6/NuGyB/7RZYzqnhmLJInQAHCXyxN6B/XGH2PaB9VzjezuTTxvqMCNTsZcOjikGY4wpZBL0ETx5LmVvP43g6QtxukKJlgp3vEB0VzPqWFI12VgydZTVbv84U36UWHPVU1cOJKYFYSC5SpW6iLJ3wcirmUeeqSe6vnHY/RqKpTUEje/tSBa199dOadwh8sw+un/2TNpnnd5hkq29HTjd4Yz7jDGmUInPS3DpLMreuoqisxdDzKHvkRfp+etzRF7cj8ZsTdXJwpKpo8y3cCplHz+DWXotnpJBiyIPSqaq//k+xDewLXjegiHn86+sS3vf9/v1tJz322Gvr33RtKVqYltb6f7OE4l9bYlkquOT99F8+q/p+Og9aXVUwy2y3Dj3h7S84sZhr2mMMYVMvB4Cx86g9E2nUnzucRDwEnpqO923PEPouV3DflE1E4c9zZdHtDN9dCf4qvkAlP/3OfiXzyD80I60/YGz5lB02RKia/aP/hp9seT6fgDh+7YN7HM7pff+Zm1yW3JNQMBp7oW5mUfHouuGHw0zxhiTWMDev6AG3/xq4o2dRDbVE1m/l8jGffgXTSewdBbeqUNX1jD5z5KpPOJfPiPj9oqvnQ9A6P5tads1FEOKh64PeCjxHW04XZmn5JL9qFKnAXe3J187zYn+KbGdbXR//0n8p9aN2P5BY3G6f7Saso+uQorGFqsxxkxGIoJvxhR8M6YQ7+gjsrme6NZGolsb8dZVElg2C9+sqmQJiMl/lkzlEW9tGbP0Wnp+80LmZWN2pz9iW/W7NxF5MlFcXvK+5fTetHbIZwZrOfcmGGbh5f6RqdRpwN7/W5d83b9UTePCHwEQOHsOxW9desjr9f5+PZ3/9QDaEaLiq+ePGJ8xxhQS75Riis88huCKeURfaiCyZT99D27CU16E//g6AsfUZnw4yeQXq5nKQ6XvX0Hpf6wasj2+a2CUiCIf/qXTk3VUxe88ccjxNWuuynyBYZp49idTkrLIct/NGwf2D2ooGtvZPpCADRLb3Y46TrJNQ7x++PWr7HFhY0yh8xT5E08Avu00il+1BCkOEH5mB123rKbvia3EW62zej6zdHcCKX7bUrq+8S8AxJ2K8y2ayiy9NvMHxvj4rdMWIvzknrRpvlSDEydnfxfx/QNJkjoOOMrBi39P+KGdVP3xLUhposh+uLYLHf91P93ff4o658s2pG2MKXji9eBfOB3/wunEW7qJvLif6PYmoi8fwFtbQeD4OnzzpiEeGwvJJ/a/xgRS/tXzmNn+ucTrL597yGMDZ89JPOnn5ifT7n/3kGOCrz0m7X18TwctZ9+I05j5G1CmDunNp/wy+Vo7wsR2tBF+aCcA0bUH0JiT2DdMQ9Du7z+V2D9MHZcxxhQqb3UZxa84NtGvauUCnJ4Iff98ceApwK6+XIdoXONKpkRkl4hsEJG1IrImW0GZzMTjwTOliFl6LeWfOfuQx5Z+aCUiwtRb345/ZR2e6UOfEJl227+N6fraEz3klJzTHYGok3wf296aHM3qT8ScngjtH7uHxhN+mtjmtoMYLoHr/Pqj9Fxv/9cyxhQuT5Gf4ImzKXvLSoovXIq3upzIhr1037qGnvs3JBqBxp2RT2SOmGxM852nqi1ZOI8Zp5KrTqX3+ucSb9ypuuI3L6X4zUuJbU3vYi4VwbQnAX1La4htbj7k+bv/39OIf/j8W7sjOCntHaLPNxC6bUtinzsy1fHxe+m98YXE/uf2I6V+NBIn3tiNb/G0Iefs+tIjAJRetfKQsRljzGQnHsE/Zxr+OdNwesJEtx4g8nIjfY+8iBT58R9TS2DJDDwVI68La7LLpvkmkapfvp6aFz6Ep66c4EWL0vZJcXrePLPpM2nvi9++bMj5pvzgNekbQjG6vvbYsNfX7gjaEUq+j+9MKZh3p/tSi+jj9V143JoqpykxMhVv6KLzK/+k+yerh71O8nqxOO0fvZvYzrYRjzXGmMnEUxokuHweZW9dRfFFy/BOryCyaR/dt62h5971RLY2olHrsH60jHdkSoEHRESBX6rq9YMPEJGrgKsA5s6dO87LmZEEls9kZv2nh2yXkvQeT/2P2k5f/2Hwewjd8VLa/uBrFuE7qXZM1+79v3X4FlZl3JcsQE8pbu+7bTPxfZ0AxN1pvgN130/uL/vP0w95vejzDfT87Fmi6w5Q8/iVY4rVGAAR+STw7yTuZRuA96tq6NCfMiZ/iEfwz56Kf/ZUnN5wsl9V6PGXCT29Df/8GvyLa/HWVthDPkfQeEemXqGqpwCvBT4qIucMPkBVr1fVlaq6sqamZpyXM4fLM7WE6keuGLLdf2It/uNqcFrTCxmn3vp2vG6dlX9VHd5F6UmSd04FUlmUtq3nJ8/Q8an7M14/Of2X0nahfwoQMi9Vk9pctP8bVuSZfRx82y2EH9+d7IfltGX+ty+6/gCt77rNvp2ZjERkFvBxYKWqngB4gXfkNipjDp+nJEjw5LmUvmUlJZechH/hdKK7W+i9dz3dt60hvHY3Tpd9VzgSxpVMqWq9+2cT8FfgtGwEZY6M4LkLqN11NTXPfHDIPu1Kf1LPUxbEt6yGsk+fSdXv3ozT0J22f8pPLsFTUzLqa2t7iL6/bRm+7UKGp/naP3Rn8rXT3IOq0nz6rwndupmWV/4m2fdqcP+rfm1X/I2+P2445MLPpuD5gGIR8QElwOjXZjImT4kIvtopFJ+9mPJ3nE7ROUvwlBURfmEP3bc+S88964i81ICGMz9lbcbusJMpESkVkfL+18CrgY2H/pTJNd+8SgKrZg3ZXv6Vc6n41gVp28TjYcr3XoN/SXVajyn/yjqKXr8Eb01i5Kri+68ecr7B27QvRuub/kz47q0Z43I6hyZTqQ1DnebeIUmT4yaA/VOI2hfl4Fv+TL1cS2xHa3Jqc7i2Cwcvu5mmVUNmpk2BcL8Mfg/YAzQAHar6QOoxInKViKwRkTXNzYd+QMOYfCQ+L4FF0ym9+ETK3raK4Ip5aChK6MltdP1pNb0PbU48DRizpwHHYzw1U7XAX905WB/wR1W9LytRmaPOW11K+edeiZT4E/2pDiFw5mxEhKrfvYmurz1GybtPovPTaf8GUf6ps4ZsO5TIU/vou3XTsPudjhByMH0qsPUNNyf2uU8Kdv/oaUK3J6YO+/60ESlNJFPxlLYLTnsfTlsI34KqIXViQ2JaU4/v2Gl4KooOeZyZmESkCrgMWAC0A38RkXer6u/7j3HrQK8HWLlypbXqNxOap6yI4PK5BE6eg3Owm+iOZqI7mojtOQh+L/751fgX1uCdUYl4rL5qLA57ZEpVd6jqye7PMlX9RjYDM7lR9vEzCJ419EGBmqeuJHjRQoBkSwXfwqlU/eaNeKrSH8OtfuIDae8rvjd05Kry129Iex9b30jr2/4ybFxOe2hIXVeSuzyOkzJVGa/vIrYl0bHDaUh0ae/+0dM0VH2bxoU/SuuXlal3lkZiNK/6FQfdhM1MShcCO1W1WVWjwO3AWTmOyZgjTkTwVpdTdNpCyt5+OiWvOQH/vGqiu1rovX8j3bespu/pbcQOdNhyX6Nky8mYUQmcMYepf3sHnZ9/iPIvvDJtn/i9ae99x0wFoPqx9ydqnQatBTjlZ6+jaFD39ZG0//sdIx4jKfVYPT97Nvk67tZ7dVw9MHDq1HcOvG7twzuthOi6A3T/9BmKLzsO/2mJqdDIo7szXiu2s422K/7KtL+9A8/U0deOmbyyBzhDREqAPuACwDrEmoIiHsFXV4WvroqiMxcR29tKdGcz0ZcbiW5pQEoC+OdX41tQg7em3J4IHIYlU2bUPCUBKn/02oz7ap67ipaLfoe29uGZmhipCr5yHsCQDuZlH16VWMcPEqNdPg/he7cl95e8fzkAvb9Zm9zmNA992i9VdEszGs085+80D+2u3v6J9MTKO62EpuW/SFz3V89T+/LHMp4r8sw+/Ktm0fWNx4j8aw+9f95E2YczLEq9r4N4Q3fG+jSTH1R1tYjcCjwPxIAXcKf0jClE4vPiX1CDf0ENGo0T23uQ6M4WIi81ENm8HykNJhKr+dWWWA1iyZTJisApddSu/zDRdQcQX/pIVeBV84ccLx4PtTs+gWd6KU1Lf5q2r/Jnr6Pjsw8mjisPDHnSMPi6xUMK2QefI1Wm6cH+2ipIjFz5Tkwfyu4bVE8Vb+hK9sCq+N6rk1Od2pf5aZjGpT9FuyLDLkKtjgMidjPKMVX9MvDlXMdhTL4Rvze54LJGYkT3thLb2Uxky34im+qRkgC+udPwz6vGO2NKwddYWQd0kzXeWRUUXXLskO3+JdXM0muRYh/+VQPF7b4FVXhKA2n9qvynzUKK/JR//pUEL1lM7Y5PDDnftL+NrRVQ6O8v0XTqL4fd77T0ooOeJuz8r/Ti+chTe5Ovw/ekJHJ9iSnM6MstHJj/Q/ZXfBONxZMJoIYyJ1v7vV+l7R23jun3MMaYXJCAj8Ci6ZRcuIzyy8+g+JwleGvKiW5tpPf+DXT/6Wn6Hn+Z6N6DBftUoCVT5qiZ2fl5ap769yHbp915ORXXXQiAd0ZZ4s+Z5VTf/S681ekLNFc/fAXi8xI8f0His/e+a8j5Zuz95JBt0ecbho0rdPfL9Pz0mWH3ayiadoMIP7yTnp8kjo+7zUZb33Az8d0daFeE2KaBR+jj+xPF79GNjbR/+C56frs2WdDZd0vmpxfjLT00nfrLIespGmNMrknAh3/RdErOX0r5O8+g+Pzj8c2eSnR3C33/2EzXzU/T+8gWItsacYb5MjkZ2TSfOWoGT//1882tpOyzZ4PfQ8l7Tx6yv+wLr6T7m/9KnMMdxZp29ztxuiJDEo6ZnZ/HUx5Mvg++ehHhB7anHVP+5VfR9ZVHk+/7bt7IMM8JAhB5eh+xYRp/Om6y5LQPdBVuffftydfxfZ34Fk6l6cSfJzb8Yg3Fbzl+yHmim5vou30LpR9YQejebUSfb6Dz648x9bdvGhrPmnr6bt/ClG9eeIiojTHmyBKfF/+8avzzqimKO8QPdBDd1UJsbyuxXS0g4J1egW/uNHxzpuGdMnkXYLZkyuQFEaH8U5mfSp/yjQsInjOP9g/ege/YaYnji/x4i/wQTx9S7k+kqh96L54ZZXR+/qG0/TPbriE0KLlK5Tu+OtlSoV/Leb8d9vj4/i6c7jCk1D7FNjYN7K/vGvKZ1rfckvbeaeujadnPAAjd9TJlH0usSZjs8K5KdO0Bos/tp+TKUxKF/u0hyj9z9pC2FAC9t2zEUx6k6LWLh43bGGOySbwefLOq8M2qQlVxWrqJ7j1IbE8r4Wd3En52J54pxfjmTMM3Zyre6RWTqs7KpvnMhFD0mmOYsedTeEoDadu9M8qpc76Md1EVUjzw3SB4/kL8S6cTOC3laToBT2UxxZcdR+knTmdGQ/qC0N6FVUzf+BFkSpDhFL99Wdr7yON7aCj/Fs6B7ozHx3e3D1kbMHWkzOkOE9/TkXwfXV1Pmzuy1d91vufHq2k+5Ze0f/BOIo/uAnfKMbazLXGOzhChe7cSemgHAG3/disHL/lDxnjUcej88iPE9nZk3G+MMeMlInhryik6ZT5lbzyFsreuouj0hUhpkMimenrvXZ+YDvxn/3Rg5iXBJhJLpsyEJyLUbv4oM1uvGbKv7JqzkzVUxW9dmjg+6KPy/70W74xyPG6NFkD5NWcjHg/VD76XkitXDGmPUPH185n657cdMpb+Wq5+nZ9/iP2Brw17fOzlg0RfbMm4r38ZnNSRtO4fPJUcsYrvSCRTTaf8koOX/IGDF/5fsuVEquiLzXR99wmiLzYT29JC11cfpe3yzMXvsR2ttH/8HjRmi0MbY7LDU15EYOksSl9zYqLO6rzj8c+bRvxAB6F/vUz3zavpvnMt4bW7ibd0TchGoTbNZyYFCWT+v7L4vHhnT6F2xyfwziwbsn/6xo/Q8/Nn6frSI8kpxMCqWQRWzcLpSf+2VPbpMwGY8uPXgldwGrrp+vpjaedCSE7ZZYxnShDtGHhysPnU4dsaxeu7iG1vTRu5Ct35cvJ1bGd74rjtbcltLa/8TfK10x1GSvw0HZ9oG9H5pYep/sd7E5/ZnTinxh16fvoM0XWNTPnBa2j/8N2EH9hO8duWJfuEpWr/yF3ED3Qz7faxPVFpjDHgFrDPr8Y/vzoxHXiwm9i+NmL7Wgm/sIfwC3sSZRx1lYlpw7oqPCWBkU+cY5ZMmYLgW1CVcbt3WgnlXzyH4tcvwX/yjLR9ntIAtbuvpvWNfyL6wgGkKNFbqr+mqe+u9F5U/mXTUceh9OozKP3ACppO+vnAToG68Jdoe8/t9P058/QG9XEAAAtcSURBVFN8pR9Zld65fVc7jcf8eNjfKfLEHnrnTknf9uRAC4f4jjYktaYqHE8mW/0LS/fe+AIdbgNTz4wyYpsTTyLG9yU6xEee20/7v9+B95ipTL3lbUQ3NIF38tQ5GGNyp39ZG291OcHlc3FCEWL17cTr24jVtxHbkbgfeapK3XqsSry1UxBv/k2qWTJlCp6IDEmk+vnmVlL9rw8kp9xSFb3uWGZ2f4GGsm9S1D+F6PFQ+cOLAai47kI6P/cPAErevwLxe6n82evwr5hJ2dVnsL/o68lzVf7yUkqvWplMpvwr64iu2Z92vSk/uYSO/7wn+T7095cI/X34xZo7rvkHkaf3ZdzX31crnrKsTv8TkwCxlxJTj82n/wriiQL4vt+vJ/L4HorfdeKw1zTGmMPlKQoQWDQdFk1PjFq19iSSqvo2IpvriWzcBz4P3top+GZW4qurxDO1NC+aH1syZcwIPKUBKB06zCwiSGmAusiXMo7WlF/zCkrecxLNZ95A2acSU4SeqSWUX/MKAMo+9wq6r3scgKK3JJKx2j2fRNv6iO/r5ODr/pg8V13sf0Cg4z/vwVNdgkbjadOF5V98JU5rHz0/H1i6J3zfwBI9mdQHvgrDLMET3dBEdGMjxAdqF9re+1cAvHXlhzyvMcaMl4jgnVaGd1oZwZPmJJa3OdBOrL6N+P52wmt2EiZRA+utq3STqyo85UUjnvtIsGTKmHEavNBzKm9dBTN2D20iCjDlWxdS8s4TCd31Mt5picWSfXOmwJwp+JbWUPXbN9J2xd8S13CHtWf2fAEJ+uj6+mN0XfvPxGeW1VDx9QuIN3QR29pK+ZdflVY75V9V9//bu//Yqs46juPvb29boB20/EqpgGwMcZLggDULc5Moiz/G1GnCH3OaTePc4sS4GGcw02VG/1ATDXMhI3NjzmXqFGckE6coGP1nbK0DBkMBB2SwFjrcZRtz0B9f/zhPy6X0dr29vT3PGZ9XctNznntz+TznHJ7z3PPrYdqTn+Xo/HvpPV7wRK2CjlT97Us5ueap/vk3H99z1pA7hfyULk4XkbFlNTlqZk+lZnZybWvvyVN0t+fpeSlPd3ue7gPJ0XSbOJ7qGcmRq9yMBqrqi9+dPZrUmRJJUc3CJmoWNp1TbtU56m5cRO2V74TaM521vgsxJ357GfVfXELHnDVM+u5yIDw1fnNygfmk7y3n1W9tAaBx7bVUTaljetutdO86RvX8qRydf2//d87ovIPctHre+Nl2PP8mNZc20bXjzENK6269jJpLZ3Ditj8AUPu+2aO8FERESlNVP47aeU0wryk5JXjif/S89Ard7SfoOnScrn1JG1Y1aQK5GQ1UNzeQm9FYsYvZ1ZkSiVj1xVMGLbeqKnLvmMTMrrsGfX/incuou3kJbzyyg5rLmpPvmtNI9ZxGAKb97XPkV22i51C+f8ie5s47IFfFqS0HyN+8kZ6DeaxxPJPXfRw/3U3PgVeYcMPCoteXiYikwczINdaRa6yjdsFMvDdcb9WRT57KfqCTrr0dAFQ1TGDCB99DbnL9W3xriRnG8nkOLS0t3tra+tYfFJGK89Pd0Ov9dykO9Nr3/8G4j8yjdnFzWf+OmbW5e0tZXxIBtV8i2ZR0rl6nu/0EPR15Jiy7BBs3vGNJw22/dGRK5DxV7NlcfSaufv8YJRERqRyrOvMIBhbOqsi/Ed/DGkREREQyRJ0pETlvmVmjmW0ws3+Z2R4zuyLtTCKSPTrNJyLns3uAJ919pZnVAnVpBxKR7CnryJSZfdTM/m1m+81s9WiFEhGpNDNrAJYBDwK4+2l3z6ebSkSyaMSdKTPLAWuBa4AFwKfNbMFoBRMRqbCLgE7gITN71sweMLOz7pc2s1vMrNXMWjs7O9NJKSLRK+fI1OXAfnd/wd1PA78CrhudWCIiFVcNLAHuc/fFwEngrCPs7n6/u7e4e8v06dPTyCgiGVBOZ2om8GLB/OFQdhb9shORSB0GDrv7tjC/gaRzJSJSkorfzadfdiISI3fvAF40s3eHoquB51OMJCIZVc7dfEeAwkG6ZoWyotra2l42s0PD/P5pwMsjzBaLrNdB+dP1dsk/J+0gQ/gK8Gi4k+8F4PPFPlhi+wXxr7+Y88WcDZSvXDHnG5htWO3XiIeTMbNqYC/Jr7kjwDPADe6+e0RfeO73t2Z9CIqs10H506X82RZ7/WPOF3M2UL5yxZxvpNlGfGTK3bvNbBXwJyAHrB+tjpSIiIhIVpT10E533wRsGqUsIiIiIpkT83Ay96cdYBRkvQ7Kny7lz7bY6x9zvpizgfKVK+Z8I8o24mumRERERCTuI1MiIiIi0VNnSkRERKQMUXamsjCAspmtN7NjZraroGyKmW02s33h7+RQbmb2k1CfnWaW+lOWzWy2mW01s+fNbLeZfTWUZ6IOZjbezJ42sx0h/3dC+UVmti3kfCw8PwgzGxfm94f3L0wzfx8zy4Vx4Z4I81nLf9DMnjOz7WbWGsoysQ1VSuzt12DrLOU8w25LI8p3t5kdCctwu5mtSClbSe14RPliWX4l7UeG5O5RvUges/AfYC5QC+wAFqSda5Ccy0iGnthVUPZDYHWYXg38IEyvAP4IGLAU2BZB/mZgSZieSPLMsAVZqUPIcUGYrgG2hVy/Bq4P5euAL4Xp24B1Yfp64LG010HI8jXgF8ATYT5r+Q8C0waUZWIbqtDyiL79GmydpZxn2G1pRPnuBr4ewbIrqR2PKF8sy6+k/chQrxiPTGViAGV3/zvw3wHF1wEPh+mHgU8WlP/cE08BjWbWPDZJB+fu7e7+zzD9GrCHZGzFTNQh5Hg9zNaElwPLScZYg3Pz99VrA3C1mdkYxR2Umc0CrgUeCPNGhvIPIRPbUIVkov2KSYlt6Zgrki8KI2jHY8kXhRHsR4qKsTM1rAGUI9Xk7u1hugNoCtNR1ymcMlpM0ivPTB3CKbLtwDFgM8kRgby7d4ePFGbszx/ePwFMHdvE51gDfAPoDfNTyVZ+SBqeP5tZm5ndEsoysw1VQBbqONg6i02xbSgmq8Lp6vVpnobsM8x2PDUD8kEky6/E/UhRMXam3hY8OT4Y/XMnzOwC4LfA7e7+auF7sdfB3XvcfRHJuJCXA5ekHGnYzOxjwDF3b0s7S5mucvclwDXAl81sWeGbsW9D56kh11lsIt2G7gMuBhYB7cCP0gwTezs+SL5olt9o7Udi7EyVPIByRI72nbYIf4+F8ijrZGY1JBv4o+7+eCjOVB0A3D0PbAWuIDl11Pdk/8KM/fnD+w3A8TGOWuhK4BNmdpDkVNBy4B6ykx8Adz8S/h4DfkfSGGVuGxpF0dexyDqLTbFtKArufjTshHuBn5LiMiyxHY8iX0zLr88w9yNFxdiZegZ4V7iavpbkYtuNKWcaro3ATWH6JuD3BeU3hruZlgInCg7BpiJcb/MgsMfdf1zwVibqYGbTzawxTE8APkRyPn4rsDJ8bGD+vnqtBLaEX2ypcPdvuvssd7+QZBvf4u6fISP5Acys3swm9k0DHwZ2kZFtqEKibr+GWGexKbYNRWHAtX6fIqVlOIJ2fEwVyxfR8it1P1LcaF0VP5ovkrt+9pKcu7wz7TxFMv6S5PBkF8k51S+QXMPyV2Af8Bdgip+5Y2BtqM9zQEsE+a8iOfS7E9geXiuyUgfgvcCzIf8u4K5QPhd4GtgP/AYYF8rHh/n94f25aa+Dgrp8gDN382Umf8i6I7x29/1fzco2VMHlEm37VWydpZxp2G1pRPkeCdvwTpKOS3NK2UpqxyPKF8vyK2k/MtRLw8mIiIiIlCHG03wiIiIimaHOlIiIiEgZ1JkSERERKYM6UyIiIiJlUGdKREREpAzqTImIiIiUQZ0pERERkTL8HyEZFW32k/UTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "# 定义网络结构\n",
    "model = Linear(2)\n",
    "# 定义优化器\n",
    "opt = Adagrad(init_lr=0.1, model=model, epsilon=1e-7)\n",
    "train_and_plot(opt, 'opti-loss2.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.3.2.2 RMSprop算法\n",
    "\n",
    "RMSprop算法是一种自适应学习率的方法，可以在有些情况下避免AdaGrad算法中学习率不断单调下降以至于过早衰减的缺点。\n",
    "\n",
    "RMSprop算法首先计算每次迭代梯度平方$\\mathbf g_{t}^{2}$的加权移动平均\n",
    "$$\n",
    "G_t = \\beta G_{t-1} + (1 - \\beta) \\mathbf g_t \\odot \\mathbf g_t,\n",
    "$$\n",
    "其中$\\beta$为衰减率，一般取值为0.9。\n",
    "\n",
    "RMSprop算法的参数更新差值为：\n",
    "$$\n",
    "\\Delta \\theta_t = - \\frac{\\alpha}{\\sqrt{G_t + \\epsilon}} \\odot \\mathbf g_t,\n",
    "$$\n",
    "其中$\\alpha$是初始的学习率，比如0.001。RMSprop算法和AdaGrad算法的区别在于RMSprop算法中$G_t$的计算由累积方式变成了加权移动平均。在迭代过程中，每个参数的学习率并不是呈衰减趋势，既可以变小也可以变大。\n",
    "\n",
    "**构建优化器**  定义RMSprop类，继承Optimizer类。定义step函数调用rmsprop更新参数。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class RMSprop(Optimizer):\n",
    "    def __init__(self, init_lr, model, beta, epsilon):\n",
    "        \"\"\"\n",
    "        RMSprop优化器初始化\n",
    "        输入：\n",
    "            - init_lr：初始学习率\n",
    "            - model：模型，model.params存储模型参数值\n",
    "            - beta：衰减率\n",
    "            - epsilon：保持数值稳定性而设置的常数\n",
    "        \"\"\"\n",
    "        super(RMSprop, self).__init__(init_lr=init_lr, model=model)\n",
    "        self.G = {}\n",
    "        for key in self.model.params.keys():\n",
    "            self.G[key] = 0\n",
    "        self.beta = beta\n",
    "        self.epsilon = epsilon\n",
    "\n",
    "    def rmsprop(self, x, gradient_x, G, init_lr):\n",
    "        \"\"\"\n",
    "        rmsprop算法更新参数，G为迭代梯度平方的加权移动平均\n",
    "        \"\"\"\n",
    "        G = self.beta * G + (1 - self.beta) * gradient_x ** 2\n",
    "        x -= init_lr / paddle.sqrt(G + self.epsilon) * gradient_x\n",
    "        return x, G\n",
    "\n",
    "    def step(self):\n",
    "        \"\"\"参数更新\"\"\"\n",
    "        for key in self.model.params.keys():\n",
    "            self.model.params[key], self.G[key] = self.rmsprop(self.model.params[key], \n",
    "                                                               self.model.grads[key],\n",
    "                                                               self.G[key], \n",
    "                                                               self.init_lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**2D可视化实验**  使用被优化函数展示RMSprop算法的参数更新轨迹。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x1 initiate: [3.], x2 initiate: [4.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAF5CAYAAACFu8BrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XmcLNdd3/3PqaWr9+7pnvXuslbbsrGEwvLYOAbjBGKbhGCSQCB+SBwB4QkkxAFMngcHEoNJeDnghOAoL0hIWBwDtjHYCMtGxpZtJFveJFmydLXd/c7We3dV13KeP6q6bs92l9G90zNzf2+/js9S1T3VI2n626dOVSutNUIIIYQQe5Ex6QMQQgghhNguCTJCCCGE2LMkyAghhBBiz5IgI4QQQog9S4KMEEIIIfYsCTJCCCGE2LOsSR8AgFLqOaADhECgtb5rskckhBBCiGtpq/d+pdQ/B34sGf+w1vqnLvY8uyLIJL5Va7086YMQQgghxI5Z896vlPpW4G8DX6e19pRSs5d6Ajm1JIQQQojd4keBd2qtPQCt9eKlHrBbgowGPqqUelgpdfekD0YIIYQQ19xm7/23AN+ilHpQKfWXSqm/dqkn2S2nll6ltT6dTCHdp5R6Qmv9yfEdkhd5N0ChUPj62267bRLHKYQQQuy4hx9+eFlrPbNTP08pdTW+v+gxwB3r36O1vmesv+G9nziX1IBvAv4a8D6l1Iv0Rb5PaVcEGa316aReVEp9APgG4JPr9rkHuAfgrrvu0p///Od3/DiFEEKISVBKPT/pY9gG92IX72zx3n8KeH8SXB5SSkXANLC01fNM/NSSUqqglCqN2sDfAB6d7FEJIYQQ4lq5yHv/B4FvTcZvATLARS8E2g0zMnPAB5RSEB/P72mt753sIe09URQRhiFhGBIEwabtzcrocZvVl1O01hv6o7HxeqsCbGiP11ci+XdoTb2+vVkxDGNNPWqPyvr+xYppmlvWWxXLsjZtG8bEP2cIIcS1sul7v1IqA/yWUupRYAi8+WKnlQDUdt4wJu0qnbsTQgghdpWt3pOVUg/v5D3WrtL77I4c826YkRHAq1/9aizLwrIsbNvGNM20Ho2Pyvgn9/X1Vp/0LzYzMD5zsNnswvrZidH4VjMam81wXKzAxtmT7dpqludSZf3s0vrxUTsMww2zUFvNYo3PcG01E3axmbPR2Gb1eAnDEN/3N4x/8pOfvNivSggh9oV9G2Re/epX4zhOWjKZzJr2qD9qj/q2bW/aH9WWZaWPs217zfZR2zTNSb98IbYlDEOGwyG+76f1qAyHQzzPIwgChsNhWkbbxvue560ZG+97npf2R23P8yR4CSG2ZU8GmTvuuIMHHnhg0ochxL5jmia5XI5cLjfpQ9kgiqI09Lium9br2xcrg8FgQ3swGPCxj31s0i9PCLFNezLICCGuP4ZhTCRkhWHIYDCg3++nwaff76+pe70eruvS7/fp9XrptvXlvvvu29FjF+J6IEFGCCEuwjRNisUixWLxmv2MIAjSEDSqR2V9v9fr0e121/Tb7Taf+MQnrtnxCbGbSZARQogJsyyLcrlMuVy+qs87HA7pdrt0u106nQ69Xo9Op5P2x+tut0u73U7H7r///qt6LEJcKxJkhBBin8pkMtRqNWq12gt+riiK0iDUbrfX1K1Wi06ns6bdbrdptVp8/OMfvwqvRIitSZARQghxSYZhUCqVKJVKHDhwYNvPE4Yh7XY7DTqjdrPZTMdG46O2hCFxMRJkhBBC7BjTNJmammJqamrbz+G6bhpyms3mmrrRaKT9UZHTZPubBBkhhBB7SjabJZvNMjc3d8WPjaIoPQW2urqahp1Go5GWUX91dZW//Mu/vAavQFxNEmSEEEJcNwzDoFKpUKlUOHLkyBU9djgcpmFndXWVRqPBysrKmv7q6iof+MAHtn18URTJ96xdIQkyQgghxGXIZDLMzc1d8UxQv99ndXWVlZUVVldX07KyspKW1dVV7rvvPgkx2yBBRgghhLiG8vk8+XyeQ4cOTfpQ9iWJfkIIIYTYsyTICCGEEGLPkiAjhBBCiD1LgowQQggh9iwJMkIIIYTYs+SqpT1Ia00URYRhSBAEa0oYhmvGR+3ReBRFa/rj41vV69tRFKXHsL5ordNt4/V4AdLx0esZb4/XV0IptaEeb48uaxyNj4phGGvqUXt9WT9umuaG9mb1eLEsa824ZVlpPWqP98eLXJYphBAbXfdBJooiPM9Ly3A4ZDgcpn3f99O+7/tr9hlt26zv+37aH9VBEKwZG5XR+Hh7vB4vozFxfbIsC9u2N4Sc0dh4vb5t2zaZTCYdH7VH9ag9Pr5Z33EcbNvGcZw1+ziOs2bbqEgAE0JcS3syyCwtLfGud70Lz/NwXZfBYJC2XddN26Mwsr49Ciau616TUDD6Y77+DWKzNxbHh4LpYBWK2JYVF9PGskws08K2bCzTxLJMbNPGTMYt00yKldaGaazpm4YRf8o3DAzDxEy2m4aBaSSzBGnbQCkjHTNG46MZCGUk+yhMw0QZCkNd2KZUfMdMxdgMx7qZD8VotmNtHzbOnmzXVrM8mtFMkV7TT2eQxmaSIj2aRYJIRxfGIk0YhWitCcMo3RZGEWEUXmiPz2RFYToWRiFBUodhRJRsC4IgHQ/CkCAMiMKIIAzSflzH7TAI8UOfIIjH/MCP90nG/SDADwKCwCfwQ4YqYDAYbAjL6wP3qFxtlmWRzWbXhJtRGR8ftUe3nx+1c7ncmvFRGY3ncrm0v75t2/ZVfz1CiN1FbWcKf9KUUulB27a95R/AbDZL1oeMnSHrODh2Bifj4GQycTsdi/sZO0PGti/0x8YdO/lUatvJWBJQrLidsW1sy8IyrRf0RizEJGmtCcI4CA19Py5BMqvo+3j+kGESfjx/iDccMgySWcyk7/lJKAr8tO953oXtw7jteh5Df4hrs+FDyHjb9/1tvx7TNMnn82m4GbXH69H4eHt87GKlUCiQy+Vk1klcNfl8ftNxpdTDWuu7duo4xt9nX4AdOeY9OSPzsptfzP3//X04dgbTNCd9OELsG0opbMvGtmzy2dykDweAMAzT4DPwXFzPxR0Ox9qj8bgeuPH4wPPoe4O0PXAH8fak3fE7LC4uMhgM6Pf7ab2dWalRqCkUCuTzeYrFYlqPxguFwoZ+qVRaUxeLxbQ4jnMNfptC7D97MshYprlr/sgKIa4t0zTJm7kd+28+DEP67oC+6zLwBmm7P+jTcwdp3Rv06Q8Ga8b6gz7dQT+pBywtLdHv9+n1emm5XLZtrwk2o1IqlTbUo3a5XN5Qj0KSzBqJ/WpPBhkhhLhWTNOkVChSKhSv+nNHUUTfHdAd9On1e0ndpzvo0Rv06fR7dHs9uoMe3X6Pbr9Pt9+Lx/tdOr0e586do9vt0u126XQ6l7XOTymVBp5yuZwGnPF6fbtcLlOpVNa0s9msnDoXu44EGSGE2CGGYVDMFyjmC1CfecHPp7XGGw5p97p0+92k7m1ad3rdpO7Q7nZptVqcPHmSdrtNu92+rNki27Y3hJtRv1KpUK1W07HN2qVSSWaGxFUnQUYIIfYopRRZxyHrOMzW6i/ouYIgoNPv0e52aPe6tHtJ3e3Q6nbiftIelfbQ46mnnqLVatFut+l2u5c83lG4GQWcUalUKkxNTa0Zq9VqVKvVdNyy5C1LbCT/VgghhMCyLKbKFabKlW0/RxAEtHtdWt02zU57LPS0abZbNLsdWp02zU4r3h5cCELNZpN+v3/R5y+Xy0xNTW1aarXamnpU6vU6mUxm269J7H4SZIQQQlwVlmVRq1SpVarbevzQH9Joj4JOi9V2Kw5AnTar7SbNTptGu8lqu0Wj3+fMmTOsrKzQbDYvulaoWCxSq9XSkFOv19P+qNTr9XS8Xq9TKpVkPdAeIUFGCCHErpCxM8zVp5mrT1/R47TWdPpdGq0Wq+0mjXaLRrvJSqvJaqvJaruR1C1Wez1OnjzJ6uoqjUZjy69DsW17TcDZqkxPT6d1Pp+X8DMBEmSEEELsaUopyoUS5UKJowcOXfbjwjCk2W2z0oyDzkorCTytBiutBsuj8aDHE088wcrKCisrK0RRtOnzZbPZNNiMws3FytTUlCx+vgokyAghhLgumaZJvTJFvTJ12Y+JoohWt8NKs8FKa5WlxiorrQYrzQbLzVWWk3ql0+HZZ59leXmZTqez9c+v15mZmUnLe9/7Xgk3V0iCjBBCCHGZDMNIF0XfxLHLeozreSy3VllurI4FnlUWV1eSepnlQZcnnnhCQsw2SJARQgghrqGs43BodoFDswuTPpR9SaKfEEIIIfYsCTJCCCGE2LMkyAghhBBiz5IgI4QQQog9a9cEGaWUqZT6olLqTyd9LEIIsZu8994PcevfeTWFb76ZW//Oq3nvvR+a9CEJcVWsf+9XSr1WKfUFpdSXlFIPKKVuutRz7JogA/wE8PikD0IIIXaT9977IX7snT/LyXNn0Fpz8twZfuydPythRuwX69/7fwP4h1rrVwC/B/y/l3qCXXH5tVLqEPB64B3AT074cPYUrTVhGBKEIUEYbKjDIKmjKB0f7R9FYdoPo4hwrB/piDCM0Ekd90MirQmjkCiK4qI1UfJYHWk0cV9rTaR1XEfR2Hh8zKP+6DVodLptNHalRrcGV0qhFChUOmYYRtpX6kLfMAyUUhgq3pbuZyhMw8QwDIxk3DCMeEwpTNPEUAamaaCS2lAGlmlhmvF+pmFgmiaWaWIYcT3qW6aFaST7W3F/ND6qbctKj09cv97+nl9h4Lprxgauy9vf8yv8g+/4rgkdlRAv3Bbv/RooJ+0KcOZSz7Mrggzwq8BPAaWtdlBK3Q3cDXB4/sA1P6AwDPH8Id5wGNeel/bdocfQHzL0/XRsOF77Q3zfZxj4yZiPH/gM/aQEyfZkHz8ICJLtfhDgBwHDYEiQtOMy2i8gCOOxIAzxA/+a/y52klJqTSC5XOMBaDshaDezLRvLNLEtKw46VhxybMtO6ngsY2XSfsa2sZLtGcsmY9vYtk3GypCxk34y7mQy8T6j2s7EY0nt2HGxbZtsxrkw5jhxncmQzThyI69r5NT5s1c0LsQuMq2U+vxY/x6t9T1j/c3e+98CfEQpNQDawDdd6odMPMgopd4ALGqtH1ZKvWar/ZIXfw/Ai190s37o0S8y8FwGnofruvS9AQPPw/O8ZNzFTdru0MMbeumYNxyuGY/rYdKOA8nVDAimaeLYyRuIlbyhjN5ILDt+E0r6VssjZ1rYZh7LNrEdC9s0sQwT24g/rdtG3LdME0uZ2KaJORpLi5GOmcpI+3E7nmlIx5Ux1o9nEpRSmCrZpox0zBiNG/G4UgoDhaGSmYvRzIZSYzMg6/qQPDaeNUn+Pbhqv++RNNwksz2RjtCMzQgloWd9P9RRun9EMquko3jWSkcX9ouSmaqkjMaCKIz3S/YPRo8b2xZGYToeRGHaD6IwLWEU4ochgU5m3KIQP5k186Mg7idtPwzw3ZAg9AinDFrdThp847CcBOUkPI9CeBiGV+33bVt2EmoyOEngiYOPQzbjkHOy8Zgzasd1NuOkYzkni+M45JN2Npsl5zjknBw5J0s+myXrZMlnc+SzWTJ2Zl/PWN37mU+gUGg2hvNDc3JzNXHt3HHHHTzwwAMv6DkKhcKy1vquzbZd5L3/XwJ/S2v9oFLqXwPvIg43W5p4kAFeCXyXUupvAVmgrJT6Ha31D2z1gCeePc5r3vK9l3zi0R/S7OiPYvIHNudksdtD6paNY+fJ5jI4lo1j2mStC+2MZeGYGRzLImPaOJZNxrSSbWvbjmlhj/pmvC1jWdhGfApB7Lx0ZgcFCsxdtSRsdwijCD8KGAYBwzBgGPp4oY8fBnhhHIK80I+3jbW9wI/3DQK8cMgwCPBCHy/wcYPh2rbv4w08+mXNSquRfmCIP1jEHzqG/vY+OBiGkQacnBOHm3wuP9a/MFbI5shlcxRyOQrZPLlslmKuQD4Xj+Wz+XRbIZ+nmCvgZCYTlLyhx8/911/hP7/3f3BodoHl5iru0Eu357JZfv5H3rrjxyXEVbTZe/+Hgdu01g8m+/wf4N5LPdHEg4zW+m3A2wCSVPbWi4UYgKPVOX7pb/wQOStDznbI2hmyVoacFddx38ZQ8sYlxMWYhoFpxP/dTFKkI9zAx/WHuMGQQRDXrj9k4Htpf5Bs7/seA99Lx/q+yyBpD4Ye/V6P1aLHqfNn6LsuA3dAzx3QdwdXdOrRMAyKuTz5XJ5iLk8hl6eYLyR1nkKuEI/n85TyBYq5AsV8gWI+3q+UL1LI5SkVCkm/gJNxLvozj594jn/0//0EX/raY/zwm36AX/rnb+OD9/85b3/Pr3Dq/FkOzS3w8z/yVlkfI/a0zd77gb8DnFNK3aK1fhJ4HZdxEdDEg8x2VLMFXnfTnZM+DCHEVWIog7ztkLcv/ib/QmmtcQOfvu/S9z36Qy9t93yP/jAZ9z16Q5e+79IbuvSSuu97uKbJSqvBibOn6A769AZ9Or0eQRhc1jHYlk0pX6BUKFIuFOOAk7QXV5f57Fe+gGkavPmN38sdt93OvZ++n9land95x3+mUixRKhSpFEtorff1aTVx/dFaB0qpfwr8kVIqAhrAP77U49ReXBj5ioUb9cf+8TsnfRhCCAHEAWkYBnSHA3pDl+7QpTscrO17g2Qs3tbx+kk9oJ2JePbMSTq9LoZSRJfxd9kyrTTYlItFqsUy5WKJSrFEJWlXi6WkLlMpXRirlMpUiiVsy96B3464ErlvvHHTcaXUw1utN7kW7rzzTn0V1sjsyDHvyRkZIYTYTZRS8do6y6aeL1/6AWO+fO4Z7v7gr9Hr93jrq97Ev3rV9wDQ8eKw0xkOaLv9tN3x+rRHxY3DUMvt0Ysinj9zila3Q6vbpt3rXvI0WiGXp1IsMVWuUCmW17Sr5TJTpQrVUoVKqZS2q+UytXKVnJOVGSGxK0iQEUKICdBa898+9xF+4S9+h+lChfd//9t55dGXpNunckWmcsVtP3+kI7qeS9vr0/J6tNwebfdCu+P1aboXxruWwZml83z1madodlq0up2LPn/GtqmWKkyVKkyVx0s1bdfSdpVaUleKJUzT3PbrEmI9CTJCCLHDlntt/vmf/jofe/qL/M2bv553v/6fUctveRutbTGUQTmbp5zNc4jpK358GEVrwk7T7W5oNwdJ3e5yxjvPY08/SbPTot3rbvm8Sqk0/NQqVabKVeqVKrXKFLVKlVq5Sn3UrlSZrk5Rr9TIZbMv5Nch9jEJMkIIsYMeeO5RfuRD/5nGoMMvvu6HeMtd37ErT9GYhkE1V6S6jVkhPwxouX2abpfGoEvD7dIcxO2m22V10KEx6NDod1kKV/jac0+z2mrQ6fe2fM58Nke9OkW9MsV0dYrpao16tUa9MkW9OsVMtcb0VI16sq1WrsrMz3VCgowQQuyAIAr5D5/8A371Mx/gxtoCv//338bL5o5N+rCuCdu0mC6UmS5c2XqhYRjEwWfQYXXQYbXfSdpdVvrteGzQodnt8MzpEyw3Vrec/VFKUStXma7WmKnVmK7WmK7WmZ6qMVOtMTNVT8brzEzVqVck+OxVEmSEEOIaO9la4of/+N187tTX+L6Xv4Zf+hv/mEJGTpWslzEt5opV5orVy37MMAxYHXRY6bdZ6bVZGbWTstxv0yTiiWePs9x8iJVWc9NF0Eop6pVqHHCm6szWptP2TK3ObK3ObDI+W5umkMtfzZcuXgAJMkIIcZX94aOf4h2f+H1Ot1eo5Yr0fQ/TMPlvf/vH+bsvfdWkD29fyZgW88Up5otTl7V/GEVp8FnutVhOws5yr8VKv81Sr0UjDPnyk19lqbFCs9Pe9Hny2VwSai6Em1F/vjaT9ufq0xTzhV15+nC/kCAjhBBX0R8++il+8iP/jUEwBGBl0EGh+PnX/n0JMbuAaRjMFCrMFCowc/iS+3uBnwacpX6LpV6L5V5cL/WaLHVbPDc4yUOPfJGl5uqmsz2j0DNXn2YuCTlz9Wnmp2eZr8+k9WytTu5avOh9ToKMEEJcRe/4xO+nIWZEo7nnc3/Gj37jGyd0VGK7HMvmQLnOgXL9kvuGUcRKv81ir8lSr5XWS70mi90mK07E06ee49Nf+hwrrcaGx1umheu5slbnCkmQEUKIq0Rrzan28qbbTrdXdvhoxE4zDYPZYpXZy1jjMwwDlnpNznebnO82WOw2aXt9CTHbIEFGCCGugq434F/f+9+33H7wMj7Ri+tHxrQ4WJ7mYPnK7/Ej1pKvhxZCiBfoK+ee5bX/42d4/1c/zRtv/UZy675NPGdl+Dev+b4JHZ0Q+5vMyAghxDZprfmth/+cn/v4/6KWK6VfMzB+1dLBcp1/85rv4023f8ukD1eIfUmCjBBCbEPL7fETH/4NPvy1h3jtjXfwX97wY+kN4N50+7dIcBFih0iQEUKIK/Tw6ae4+4O/ypnOKm//th/gn33jGzCUnKkXYhIkyAghxGWKdMR7Hvow/+7+32OhVONPfvDnuevgLZM+LCGuaxJkhBBiC+NrXRZKU9RyJR5dfJ7X3/IN/Orrf2RbX6gohLi6JMgIIcQm1t+h90xnlTOdVf7e7X+d//LGfya3nBdil5Ags49orYm0xo8CgigiCJN61I9CwiiM2zpM+lFah1FIqCOCsXYYRYQ6IhrVWid1PBYlP3M0pjVoLoxHyWNGxwY67evkmDU6eVz8v/i1jL0uNt7ye0Rx4c1k9L6iRv9TSVvFeymlMJQC4nrUN5SR1AqFgVJgKANzNG4Y6X6jMVMZGEbcN9PaxBprm4aBNVZbKtlumHHfMLAMK67NuLYNKz02MVmb3aEX4DMnHpN/PkLsIhJkrlCkI7wgwAuGuKHPMPDxAh839PGCYbwtHDIMAtxgyDAM8JL9Rm0/HQsYhkHa98OkHwV4QUAQXdjuR2G6PUjaQRSu60cXfdPfjUZBZH3oiMfUhv3Gjb/W0febrA9H6/fbCxQqDjWmFZck+Iz3x+tMMp4xLWzjQt8x7XQ8Y13oO6adjNk4po2T1PE+GRzLwrEyZJNtGctO2vG262VR62m5Q68Qe8K+CDLDMGDge/STMvA9Bv4wrgOPftJ2gyFuMBofMvCHuH4ylmwb9d3Axw2GeMk2L+kPw+CqHLNlmGRMC8eyN33zsU2TjGmTCaFgOGQsK34zGyuWYZIZvcmN12p8BmD0yf/CuKmMdOYg7SsDIxm3lIlhxLMOZjJujmYh1s1MjGYnlFIYrJ/dWFcn+4yHlp0yHm4iLswQadbWo9klnewzmoVaMxOVtMNkxinUo1krncx0xfsFyXgwPtOV9INRHY368QyZP9b3owsliDb2h1GAH4YEKqLv9/DDcEMoHoZ+UseB92pIw49lk7MyceixMmTtOPxk7bifS8ZzdrwtZ2XI2U4yFtd5O0POcshnHHK2E7fteHvedrDNyfyJ+vTzX0UpA62jDdvkDr1C7C57Msg8vnSC29/9w2l42c4faMswL/yxTf7wpn+MI0XVLpDNTeGYFjkzg2NaZNM6+SSbBI9s0s6kYxYZY6w91s8YFhnTvG4+1e4WKglUKLhev8kk0hHDMA5AXhgwjEazhBf6aTuM227o44Wj8WTmMQxwwyFeGDBIajeZoXTxabq99EOD61/4kLDd/07ztpMGm7ztkM9kKYz37SyFzFjZol/MZClmchSdHJktAlKkI/7Tpz/Af/jU+5jOl2m5fbzQT7fLHXqF2H32ZJDJmxm+feEl5JMgMqrXtM0LY1nTTrdlTZuslcE2rte3M3G9MpRB1jLIYk/k5/tRmAae/mgWNPTj2dFgyCCM69G2C7WXtntJuxMNON9txLOwQ4+e79L3vcs+FtswKTq5ONgkAcc2LZ5cPsVyv82NtQW+8+a7ONNZ5S+e+TJNt8t0vsw/uuPbuXXmMM81zlNycpSc/JahSAixM5TWe2v9AMDXTR/VH33jz0z6MIQQu0ikIwaBTz/w6PkevcCj67v0fI/uaMx30/GuP9rucqrb4PHmaYIopOwUiHREd+he1vqqrGVTyuQpOjnKTj4tpeyFdiWbpzRqOwXK2Txlp0Al2WdSp9DE7jP9ju/ddFwp9bDW+q6dOo4777xTP/DAAy/oOQqFwo4cs/zXI4TYFwxlULAdCrbDTO7yHhPpiHd/5c+59+RXuKE0wz2veQsvrR0C4nVV/cCj63t0fJeOP6Dne7SHg7TfGY7Xbrwt9FlunKXt9Wl7fTre4JLHkbedJPAUKGcLVJw81bRdoJorUHYKVLNxqeSKabuYyclVVOK6JkFGCHFd+aOnH+KXvvAhTvdWyZgWXhjw3TfcxX/8v76fop1N91NKxetr7CxzVLb988Ioouu7tP0BneGA1nBAx4/rdlJaw35SD2gP+yz1WhxfOUPL69Fye8mtCzZnKiMONbki1WwxqQtxO1tgKldiKhePT2WLcTtbpJorYMkpdrEPSJARQlw3/ujph3jrZ36PQRjfH8YLA2zD5NsPvXRNiLmaTMOg4uSpOPltPT7SET3fozns0/L6cb2u3fDifmvYZ6Xf5unVMzQHXVpu/6Knx8pOnloSdGr5MrVckVquNNYuU8+X1ozJaTCx28i/kUKI68YvfuGP0xAz4kchv/SFP+F7bvzGCR3VxRnKoJTJUcrkOFy8sku/Ix3RGg5oej0aXp+m16M57LPqdWl6fRpeLy5uj+Vei6eWT7Ey6NAbuls+Z9nJU8/HAaeeL1PLlZjOl5OxuIz3C5lrExCFGJEgI4S4Lqy4XU73GptuO91b3eGj2RmGMphyCkw5BW64gsd5oU/D67Hqdln1eqx6PVbcDg2vx4rbZdXtxr/P9gqPnHuO5X5ry3ts5W1nQ7iZKVSYzpeZLlSSdlzX82W5CkxcMfk3Rgix731u8Wl++BO/ueX2g4XaDh7N7ueYNvP5KvP56mXtr7WmF3isuF1W3A7LSdBZdjusuJ20vdhr8vjSCZZ77TX35xlXzRbSYDNTqDBdqDBbqDBTqKZjo3bedq7myxZ7lAQZIcS+pbXmPY99nHc8/EEOFmv89B1v5N1f+fM1p5dyZoa33fldEzzKvU8pRdHOUrSzHC1NX3LNnCg/AAAgAElEQVR/rTVd32XZ7cRlEIefZbfD0qDNktthORzw+NJJlp57hKbb2/R5ipkcM4UKs4Uqs8Uqs0nAmS1WmStUmStOMVesMl2oyMLmfUyCjBBiX2p6ff7FA/+Le09+hdcffQX/6ZU/SDmT40ixnl61dLBQ4213fhffc+M3TPpwrytKqXTdzw3l2UvuPwyDJOR0WHLbLA7aLA/aLA06cTtyeWLpJJ987hFam4QehWK6UGauOMV8cSqt50tTzBdrzJfisRkJPHuSBBkhxL4wfln1TK5MEIV0fJd/9w1v4i0v/tb0Xivfc+M3SHDZYzKmxYHCFAcKU5fc1w38OOz02yy5bc73W5wfjOoWi70mXzn3LMv91obL2hWKmUKFhVItCTlTLJRqzCVhZ6FUY6FYYypXlHv37CISZIQQe976y6oXB20U8NZXvJ5/+pJvm+zBiR2VtWwOF+uXvMIriEKW3Q7n+q005JzrtzjXb3Ku3+JUa5nPn3qSlUFn058xX6zFwSYtdRZKNQ6Uaxwo1ZktVmV2Z4dIkBFC7Hm/9IUPbbisWgO//9Rn+VeveP1kDkrsapZhXtaCZi/0WRy0Oddvcrbf4lyvydl+Mw48QY8vnD3OuSdXcYO1i5cNpZgtVDlQjgPOwfI0B0pxyFko1zlYrjNfnJL78lwF8hsUQux5W10+vV8vqxY7xzEvPcOjtabh9TjXb3Km3+RsEnbO9Bqc6zc5vnKGv3z2EbrDtV9XoVDMFqscLNc5UIrDzX/5+e/GsuSt+UrIb0sIsaf9n+N/teU2uaxa7ASlFLVskVq2yEuS7+raTGc44EwScM72GpzuNTjTb3Cm1+Bry6f43OkneY+EmCs28d+YUioLfBJwiI/nD7XWb5/sUQkhdrtBMOTfPPg+fu+pz3BLZZ4T3RXcsXuTyGXVYrcpZXLcmslxa3Vh0oeyayilTODzwGmt9RuUUjcA7wXqwMPAD2qthxd7jokHGcADvk1r3VVK2cADSqk/01pv/TFLCHFdGb8i6WChxlte/Br+4OmHeKxxin/58u/kra94PR989vNyWbUQe89PAI8D5aT/y8B/0lq/Vyn1HuCfAL9xsSeYeJDRWmugm3TtpGz9LWdCiOvK+iuSTvVW+beffz95M8PvfvuP8dpDLwXksmoh9hql1CHg9cA7gJ9U8TXt3wZ8f7LLbwP/lksEGeMaHuNlU0qZSqkvAYvAfVrrByd9TEKI3WGzK5IAyk4uDTFCiD3pV4GfAqKkXweaWuvRF3edAg5e6kkmPiMDoLUOgVcoparAB5RSt2utHx3fRyl1N3A3wCFZwCfEdWOrK4/O91s7fCRCXEcGPuqRsy/0WaaVUp8f69+jtb4HQCn1BmBRa/2wUuo1L+SH7IogM6K1biql7ge+A3h03bZ7gHsAvm76qJx6EuI6Uc+WWHY33pRMrkgSYtdb1lrftcW2VwLfpZT6W0CWeI3MrwFVpZSVzMocAk5f6odM/NSSUmommYlBKZUDXgc8MdmjEkJMWqQjfu0r97Lsdlh/M3i5IkmIvU1r/Tat9SGt9THgHwB/obX+h8D9wJuS3d4M/PGlnms3zMgsAL+dXIJlAO/TWv/phI9JCLHDxq9MWshXqTlFHm2c4rtvuItvWbiVd335z+SKJCH2v58G3quU+vfAF4HfvNQDJh5ktNZfAe6Y9HEIISZn/ZVJZ5I7pH7vjd/Eu1/1gyil+P5bXjnhoxRCXAta608An0jazwBX9Cll4qeWhBBiqyuTPnvuSfmWYSHERUmQEUJMnHxXkhBiuyZ+aklcWhhFeFHAMAzwowAvqYdRiB8GDNe0Q4IoZBgFBFGIP1aC8VrH7TCK4lpHBFFIMKqjiEhHaX/UDnVEGCW1joi0TsciNJHWaK3T8QiNTvbTQKQ10Vhf6yiu0ejkWrTx9qi/FTW2DFSpuD/6AK+SrUoZGCpuG2vaCqUMTKUwSMaVwlAKA4VpGJjJ/qaK26MxSxmYhpm2DWVgGQaWYWKpuDaVgW2YmEZc28rEMkxs40J9oVhr+6ZFxjDJGFbatg0Lx7SwDYuMaeEYFqax9z+LPN9ZxjJM/CjcsE2uTBJCXIoEmSsQ6Qg3DBgEwwslHK7pu6HPIPTTthv4uOEQLwxwgyFuGOCFPm7op/UwjMPJMPLT7cMwSMNLqKNLH9w22WNvvKM3X8swMJWZ1MbYWPKGPfaGbk+VcEZ9I95uGApDGUkoWDs2HhZG2yH+0jWlVBpMlGLNKQW14bqVtQFH6zj8jMZ0EqhG/9xG4WpURzoiikaharQ9HouDWphu9xsd/MgnjOIwFyUhLhgLdEEaBuN+HBLjYLjZG/TVYiojDTWOaZMxLbKmRcawcUwr6ds4pj1WW2StDI5pkTUzZC2bXFrH++WszIViZtb0s6aV/nN7oe47+Qj/z6f+J5YyUIZiGAXpNrkySQhxOfZlkAmjiF7g0fVder5HL/DSur+u3/M9+kGyLRjSDzz6/pBe4DEIhvTTkOIxGPtCuithGybZ9E0mQy55Q3FMi3y9SsHO4Fg2jp3BsTJxbdtkLDvpx+1Rcay1/YxlYZtxnbFsLNNKxuJx2zKTtoVljsZNTMOU9Qc7QGtNGIX4YTxrFoQB/qgEIX7ox7NpQbxtGPgMgwA/jOu4HxdvVPtD/DDA8328YJiMxeNeMExqH3fo0V5t4YUd3DAO0m44ZBgGuKG/7ZA1Cjj5JNzkrQx5yyFvx3XBctKxgh33n2ie4d4TX2bV6zGdLXFzZZ7Pnn+KW6sL/Pqrf4jHV0/zy1/8E7kySQhxRZTWe+/ecvVsUX/Lwm10gziodH2Xrj8KLu4VBY6MYZG3MhRsJ/mjHNel2Tp5J0c+45B3cuQyDoW0zpLLZMnZDjnHIZfJks9kyWYycW075DJxydoZLHNf5kWxDwRhwGA4xPU9BkMP1/foD13c4ZD+0GUwdBl4yZjv0fNcBkOPnjdgMPToewP6Sd1ZXEmC/+gDQvxBYHyW5VJypk3BzlK0sxRtJ6njUrKzFGyHkp2jZGcpZi6Ml+wspUw8Xs7kKFiOhHSxJ839jx/ddFwp9fBFbi531d354pfpT//PD76g58h/0007csx78h22PRzwyOrJ9I/d4SNHKGULFLI5itk8pWyeYjZPwYn7RSdHMZcnn8lSzOYoODkK2RyFTA7b2pO/AiGuCsu0KOUsSrn8NfsZfhDwdT/99zndWNqwrVas8PbvuZuu26fnDei4fbpjpXVuicVBi2fai3SGLt3AZRBsvLppPUOpOOzYOcqZLGU7RzkTl1Iml/YrmXxS5yhn8mvqjHwAEWJP2JP/pd5+5Gb+8u3/fdKHIYS4DLZlcaaxvOm2RrfN//3X33hFzxeEAV1vQGfQpzPo0XF7tPs92kmd9gc92oMunUGf1qDLucVlvtY8S8d3aQ8Hl1x7lrMyVDN5KkmwqThxu5rJU3UKVJxcun3KKVB1ClSdeLtlmFf0moQQ27cng4wQYu9wfY+8k6XnDTZsO1SfveLns0yLar5ENV/a9jFprem6A9qDLq3+WBl0afY6Sb9Ds9+l2e+weuYcp7sNHhuepjXs0/Xdiz5/2c5RdeKAU3OK1LIFppwCU2PtulOkli2m2x3T3vbrEeJ6JkFGCHFVve+z9/EL77+HUyuLzFfrZCybnjfAMk2C8MLi4lzG4ef+7t0TOUalFKVcnlIuz8HalYepIAxo9bs0eh2avQ6NXptmP64bvQ6r3Rar3Tar3TbL587xTGeRVbdL5yIBqGhnqTmFNNzUs6NSimunyHTSns6VZB2QEAkJMkKIq+Z9n72PH//t/8Bg6AFwthmfUvrR172JO4+9OA04h+qz/NzfvZu/982vm+ThbptlWtRLVeql6hU9bhj4NHptVjotVnttVjpNVrqtuN9tsdJtsdxpsnT2HF9rnmXV7Wx58YJjWEznSmnYmU7KTG68XY7buRK2nO4S+5QEGSHEVfML778nDTHj/vQLn+Kd3/fjeza4XC0Zy2auUmeuUr/sx/S8AcudJsudJitJvdRuJmMNljtNzp0+w5PNcywP2nhbXCU25RTWBJ3ZXJnZXJnppJ7NlZnJliX0iD1HgowQ4qo5tbJ4RePi0gpOfKXl0emFS+47Wvuz1Gmw1F5lsdVI2g0WW6ssthucP3mKR1ZOsuR2tlzrU3OKzOWTcJMrM5erMJvUc/kK8/kKs7kKBdu52i9XiCsmQUYIcVUcP3cS0zTWrIMZ2c6iXnHlxtf+vGj24CX373sui+1VltoNzrdWWWyvcr65Egee1gpnT57ieGuRpUF70/sBle1cGmzm8hUW8lXm89U19UyuJFdxiWtKgowQ4oqNL+g9VJ/lu77+1fzvT30Ex8pgqgAvuLCuY5KLesXF5Z0sx2YOcGzmwEX301rT6HU431rhfGuVs40lzjaXOddc4UxziTMnTvLZc8c5328SrLus3VCKuVyF+XyVA4ULIedAYYqFpJ7PV+SqLbFtEmSEEFdk/YLekyvn+fWP/gFH6vN8+KffzV899ci+WdQrYkopasUytWKZFx+8Ycv9oihiudPkTHOJs41lzjSW09BzurHIU6dP88kzT2x69dZ0tpQEnSkOFOJyMKkP5KdYKFRl7Y7YlAQZIcQV2WpBb6QjjkzPc2R6XoLLdcowDGYrNWYrNV5x9NYt9+sM+pxtLnF6dYkzjSVOry5yurHImcYSJ06e4K/OP0VruPa+QwrFXL7MwUItDTmH0naNQ8UaNacgl6RfhyTICCGuyFYLd0+vbvwKAiE2E6/jOcotC0e33Kfr9jm9usip1cU19Ymnn+Grq6e47+QjuOsuTc9ZGQ6NBZvDxTqHi3WOJPVsrixBZx+SICOEuCK1YpmVbmvDuCzoFVdTMZvn1gPHuPXAsU23a61Z7bY4uXqeUyuLnFw5x6mV85xYOc+J55/jkRMnWXG7ax6TNW0OFZKAU4oDztHSNEeK0xwp1alm8hJ09iAJMkKILY0v6j1Ym+XlR25ipdvCUIpI63Q/WdArdppSKr0p4Vansfqey4nlc5xYOcvzS2d5fuUczy+d5bnnnuFLK8/T8Hpr9i/ZWY6WptNwc6Fd51CxJguSdykJMkKITa1f1Htq9TynVs/zyltfwQ+88jv5xT/+LVnQK3a1vJPltoPHuO3gsU23t/pdnl9OQs7y2bT95PPP8rGTj665uaBCcaAwxbEk3MT1DMdKMxwrTVNxrt03yIuLkyAjhNjUVot6Tyyf5ftf9Z18/6u+cwJHJcTVU8kXefmRm3n5kZs3bIuiiPOtVZ5bOsNzS2d4NqmfPv4UHz35CMtuZ83+U04hDTXHynHAuaE0ww3lGaazJTlldQ1JkBFCbEru0iuuZ4ZhsDA1zcLUNN98y8s3bO+6fZ5bOsuzi6d5duk0zy6e4fjXvsbnl57lj597eM2p16Kd5UXlWW4sz8Z1ZS7tlzK5nXxZ+5IEGSHEBlEUUcrlaQ96G7bJol4h4sXItx++kdsP37hh2zDwObF8jmcWT/HM+dM8vXiKpx5/goeXnuWDzz6M5kLImc2VubE8x42VWW4sz/Gzw39CJpPZyZey50mQEUIAaxf2Zu0MA9/DNAzC6MKdWmVRrxCXlrFsbpo/zE3zhzds8/xhPHtz/iRPnTvB8XMnefLJr/FnJ76MG/i83ZYFxVdKgowQYsPC3oHvYZsW/+hb3sBHH/msLOoV4ipx7MyWC5A7g76spdkGCTJCiE0X9vphwEcf+SyP/sc/mNBRCXF9KeXkyqftMCZ9AEKIydJac3Ll/KbbZGGvEGK3kxkZIa4za29yN8Ph+vyW+8rCXiHEbidBRojryMab3MXfYXPn0Vt5/Oxza04vycJeIcReIKeWhLiObHWTu6Vuk3e/+ac4XJ9DoThcn+Pdb/4pWdgrhNj1ZEZGiOvIxW5y9/e++XUSXIQQe44EGSH2sfVf+pixbTx/uGE/WQsjhNirJMgIsU9t9qWPEH/53fidRWUtjBBiL5M1MkLsU1uth5kqlmUtjBBi35AZGSH2qa3WwzS6bZ5995/s8NEIIcS1ITMyQuxTU8XSpuOyHkYIsZ/IjIwQ+8T4wt5iLk9n0MNQikjLehghxP418RkZpdRhpdT9SqmvKqUeU0r9xKSPSYi9ZrSw9+TKeTSazqCHaZi8+dVvlPUwQohdRymVVUo9pJT6cvLe//PJ+O8qpb6mlHpUKfVbSqlLfh34bpiRCYB/pbX+glKqBDyslLpPa/3VSR+YEHvFZgt7wyjkY48+KF/6KITYjTzg27TW3SSsPKCU+jPgd4EfSPb5PeAtwG9c7IkmHmS01meBs0m7o5R6HDgISJDZZaIowg8D/DAgiELCMEzbfhASRqMSEUYRQRQSjfUjPao1YRSitSbSEVqT1JpIa7TW6eXBOjktojc5ntGX3Y++9l6hMAyFQqGUwlAGSpHUCtMwMZTCNAwMZWAacTEME8sw075pmJiGiW3F47ZpYZoX2rZpYRgTn8xc45R86aMQYg/R8R/3btK1k6K11h8Z7aOUegg4dKnnmniQGaeUOgbcATy4yba7gbsBDtfndvS4Ji2MQvqex2Do0h+6uMMhA9/DHXpx3/cYDId4/hDXH+L5HgN/1Pfw/CFe4Me17+MFQ4aBj+f7DIOkhAFDf8gwDPADn2EQMAx9giDATwJLpKNJ/yp2DUMZSagxsS0b27TIWBa2ZZMxLTJ2Jq4tm4xl49hJbWVwbBvHzuBYcZ21HbJ2hqydwUnajp0hl8mQy2TJ2Q7ZjEPOdsg5DvlMlj/70qf55Q/9NqdXF6kVy5sGPZCFvUKIiZpWSn1+rH+P1vqeUUcpZQIPAzcBv661fnBsmw38IHDJ5Sa7JsgopYrAHwH/QmvdXr89efH3ANxx7Lat/m5PVBRFdL0BXbdP1+3TSequ26fnDuh6A3rugJ43oOe5ce0O6A0H9D03abv0vQGDJKQMPA8v2Hgn1svlWPEbZMa249qycSybTPKGah7vUsIig4lNARsTBxMLAxsTe6y9vjYxsMaKmRa1pm2MjSlUMqaS3vqapAes+X/SUeDCjM26EY0m2tCO6whNmPRDIkI0UVLH/bgdEBEREawpYdr2CQl0hB+EBEHE0AvT7R4hflI8hvgMcG8q0up3GQZjgXJdsHwhVrqtTcdNZXBwapaf/N/vouDkKGZzFJwceSdL0clRyOYoZvMUs3lKSV3M5ik6uV034ySE2FlRs0/vQ194oU+zrLW+a6uNWusQeIVSqgp8QCl1u9b60WTzfwU+qbX+1KV+yK4IMkny+iPgd7XW75/EMXj+kPagR6vfpdXv0Ox3aQ96cUnbXTqDPm23R2fQpzPo0Rlrd73BZf+8XMZJ31QKTo58Jovz7IA5MuSpkMMmT4Yc9pqSxUrquL1Z7WDhJOHECFS8CunyD01cbccvvjlCMyTEI8AjwMXH3aIe4PPv+ChN3A3PE4dLhUtAxrSYLlU501zi+PmT9JJwfLmKTo5SrkApl6eULaTtcrZAORf3y7kC5XwxrnMFqvkilXyJSjLm2Jkr/U0JIa5DWuumUup+4DuAR5VSbwdmgB++nMdPPMioeIHDbwKPa63f9UKey/U9mr0OjV6HRq9No9em2esmdYdmPy6tfjftx8Gli7vJ98+sO07K2eSPefKHvPSszwFKFKlTwqGAQ5FMWhdxKJChkLTzSTuPjTk0YAh0XsgrFvuBgUpC6OX95/hWPrTpeEDIs7w97oRAc+32kIg+Pj2G9BnSxaO7rt0b1Z5Hx/PoNod08Bjc5LLYWknDfcftp+uXtpK1M1TyRSr5ItV8iWqhRDUJOqP+VKHMVGFte6pQlhAkxD6nlJoB/CTE5IDXAb+slHoL8DeB12p9eesZJh5kgFcSnwd7RCn1pWTsZ8cX/Kx3avU8b/6vb2e116LRbbPabdHodegPN35KHVFKXfgDmi9Ret7nFoqUqFMhR4Us5TXFoTRWF3QGY6BkZkNMxAd4hHfycc7Q2nI9zAEqF30OE4MSDiWcKz+AdbNKEToJPx5tPNq4dHBp4dJJ+i3fpdUa0G55tOizeDTgybMn0g8QFwtC+UyWWrEch5timVqhQr1UoV6My3SpSr1UpV5K2sWKhB8h9pYF4LeTdTIG8D6t9Z8qpQLgeeCzyYUc79da/8LFnmjiQUZr/QCMLX64DI1eh8dOPU2tWGb+pMGLOcgUOarrSoVsUucoaQejp6B3jV6IENfIB3iEn+JPGOBvuU8Om5/htTt2TAYqDf0HLvdBz19oRmg6eLQY0ExKC5cmAxr0aQ4HNFddGqt92jcNeezU06x0WzR67S0DUCVXZLpcZbpUZaY0xXQ5rmfKF8pseYqZco2pQim92k0IsfO01l8hvrhn/fgV55KJB5nteGk0y0fO/dCkD0OIHfFOPr5piDFRRGgOUOFneC3fzcsmcHTbY6CokKVCliNMXXznsdmgkIgWLiv0WKHPCj0a9Fmmx8qgz+qgT6OW4ZnFUzz09GMsd5qbXm1nmSaz5VpcKlPMlmvMVerMVWrMVmrMVeL+fLVOwcld5VcvhLia9mSQEWI/Gz+NdIAKp9n8qqQIzcnRmpjrhIlBjTw18ty81U6PX2hGaJoMWKLLMr20Xg57LDW6LDV6nD+6yiMnjrPYbhBG4YanKzo55qp15it15qp1FqrTzCf9+ep00p+mlMtfk9cshLg4CTJC7CLrTyNtFWLg0mtiRDzzMwo+t261U3LKaxR6ztNhiS6LdDlPh0Wvy9L5LisV+PLzT3Lvlz6z6Xq8opNjYWqGhalpDlTjeqE6zYGpGQ7UZjg4NcNMeQrTMK/Z6xXieiRBRohdZKvTSIq1dzfe6TUx14Px0PNiNrnp5pNxpdF0GXKeTlrO0eG81+HcuQ5LBY9PP/klzjaXCcK1MzyWaXKgOsPB2iyH6rMcrs9zqDbH4bG2zOwIcWUkyAixi5zZYgZGAweppKeb9tqamP1EodKrv25ieuMOT8dVhGaFHufocJY252hzJmxzZqXN+brioeOP8YHP3b8h7FQLJQ7X5zlan+dwfY4j0/Mcrs9zZHqeI9MLTBVKO/Aqhdg7JMgIMWHja2IUKr1r8biDVHiQfzGBoxPbZaCYocgMRV7GwtqNyexOSMQiXU7T4hRNTtPidK/FqV6L48FJ7v/q5+mtu9FmJVfkyPQ8R6cX0vrozEJay+Jkcb2RICPEBK1fE7NZiJHTSPuXicECZRYocxeH1248E//70GDAKZqcoMlpmpwYNDl1ssnx8CR/8djnNqzXmS3XODYTh5pj0wc4NnOAG2YPcMPsQeYrdfn6CbHvSJARYoL246XV4upRY+t2Xr7+jj1J0FmhzwkanKTJCRo8325w5qDiweOP8kcP/sWay8+zdoZjM3G4edHswbjMHeKm+cMcqs3KQmSxJ0mQEWKCtloTcz1eWi2unEIxTYFpCtzJoQsbkkvQfUJO0eJ5VuOQ4zd47kyDk8Y5PvXEF9ectnKsDDfMHuCmucPcNH+Ym+ePcMvCEW6aP0KtWN7hVybE5ZMgI8QOG18TsxW5tFpcDTYmN1DjBmprN5yKZ3PO0+U5VnmGFZ4JVnj2zApP6uf58698Fj8M0t3rxQo3zx/h5oU43Nwyf5RbDhzl6PS8zOKIiZMgI8QO2o1fNyCuTwrFPCXmKfFNHL2w4SwERJygwTOs8DTLPN1d4Xkj5N4vf4b//akPp7tmLJub5g5x88JRbjtwjFsWjnLrwlFuXjhM1t7Gd3oJsQ0SZITYQbImRuwFFgYvos6LqPPt3BIPJldaNRjwDMs8xTLHg2WOn17mEf8p/uThT6brcQxlcHRmgdsOHOXWhWPcduAYtx28gVsXjpJ3shN6VWK/kiAjxDV2Od9cLWtixF4xRY6v5zBfP36V1SK4BDzDCsdZ4km9xPHFZZ62z/KxRx5KT1MppThSn+fFB2/gJYdexO2Hb+T2Qzdy49whLFPejsT2XNa/OUqp24CDwINa6+7Y+Hdore+9VgcnxF53OaeSQNbEiL0vi8VLmOMl43dFPh0vOH6OVZ5MAs5Ty0s85ZzlY48+mN4MMGtnuO3ADXGwOXwjLztyM7cfvpFqXm7+Jy7tkkFGKfXjwI8Rr4P/TaXUT2it/zjZ/IuABBkhtrDVqaRxsiZG7Gc2Jjczw83M8PrR4GnwCDjOMo9znq/653iyEHDvlz/D7zzwkfSxR+rzvOzITdx++CZedvgmXnbkJo5OL6CUmshrEbvT5czI/FPg67XWXaXUMeAPlVLHtNa/RvwVMEKILVzsyiQFsiZGXLccLF7KPC9lHvg6+Gp8JdUiXb7KeR7jHI+vnOfxzAk+8qVPo3V8YraSL/LyIzevKbcsHJFTU9exy/knb4xOJ2mtn1NKvYY4zBxFgowQmxqti9lqTYx85YAQGykUc5SYo8S3clM8eBb6DHmCRR7jHI/1z/G41+c37/8grj8E4lNTtx++ia87ektSbubFB27AsTMTfDVip1xOkDmvlHqF1vpLAMnMzBuA3wL5GCnEepdaFyOnkoS4Mnky3MmhCzf9eza+RPxplnmUczzmn+XxjMcf/NV9/Ob9HwTiS8NvP3wjd/3/7d1pcBz3eefx74NjgMFJHAR4EwRIggQpixRJXdRhWbIjey1q5aNseR0nlKtUu5vsWrZrs9Z6bW+yeaHEW1l7N0kltKNSpeKVL4mxE9uRbUW2SpYoWaIkkxR1UOIFgiQIAiCuGQww8+yLGcAgRQIgBKCnZ36fqq7unp5pPF085of+H93cxpZV69na3EZL4zI1S+Wg6QSZTwGjE19w91HgU2b2t3NSlUiITdYvZqmakkRmRREFtNJAKw18mHfBq+nRf0fpYT8neXm0g/2RIb711E/Y9fijQPrJ4mOhZltzG1ua2zRrcQ6YMsi4ezuAmX0duM/HGirTx341h7WJhMZ0hlgbqDlJZA4VYOMzGd/BBngt/YTx1znDizQBi4wAAB1PSURBVJzgxcF29vWe5av/9Pfjc96sblzOtpY2trZsYFtzGxuWNau/Tchczp9WP/BDM/u4uw+a2e8AX3b37XNUm0goaIi1SPYqpID1NLKeRj7BVdAOgyR4mQ720s6Lp9v5eew5Hn76MQDKIqVsaV7PtpYNXNOykW0tbdRVLgj4KmQy0w4y7v7fzewTwC/MLAEMAF+Ys8pEQkJDrEXCpZwI19PE9TQB4H3OcXp5gXb2Jtp5Kd7H13/yMMlUep6blsZlXLN6I1e3bOSG1k2sXrRcfW2yyLSDjJndSnoo9iCwGLjH3V+bq8JEwkJDrEXCzTBWUMMKatL/To9AjBFepoPnOc7e0+08NvgM/+9X6WnTGqpq2d56JdtbN3FD6ybWLWlSsAnQ5TQtfRH4krs/ZWZXAN8xs8+5+7/OUW0iWU1DrEVyV5RirmXl+AM1fcB5i7Ps4Sh7+o7y3KH97P71EwDUVy7ghtbN3LBuEzet28zaxSsVbObR5TQtvWfC9j4zez/wCHD9XBQmks00xFokvxhGC/W0UM+/Ywve4xyjlz0c4Zn+o+x56wD/+Hw62CysquGG1k3cuO4qbl5/lYZ9z7EZd81295OZ5iaRvKMh1iL5zTBWUsNKavgYm/HudLB5hiM83XeEPRPu2CypWciN6zZz07qruGn9VayoXxRw9bnlHY0xc/fYbBUiku00xFpELmVisPk4m/Ee5zDdPM0Rnu45zOP7n+M7z/wUgKaFS3h32xZu23gNN7dtoSpaHnD14abB8iLToCHWInI5DKOZOpqp45Nswfud1znDrzjMr84c5pFnH+ehX/4TRYWFXLv6Cm674hpu23gN1/sNaoa6TAoyItOgIdYi8k4YNj4T8T1cw0g8yfMc5xfJQzw5dJr/8f2/5U93f5Ou+z5EdbV+IbocCjIikxhrTjqhIdYiMouKKeQ6mriOJu4/DqfoZ3/ypELMDCjIiFzCdJqTNMRaRGbDIipZRGXQZYRSQdAFiGSrqZqT1JQkIhI8BRmRC+xmH9fwtUmbk5ZSzZ9zh5qSREQCpqYlkQnUnCQiEi66IyMygZqTRETCRXdkRDJ2s2/K5iSNTBIRyS4KMiL8tknpUtScJCKSndS0JMLkTUpqThIRyV5ZEWTM7EEz6zSz/UHXIvllOiOUNDpJRGR2mdlyM3vCzF4xswNm9pkLjn/ezNzM6qc6V1YEGeAh4Pagi5D8MtacNFW/GIUYEZFZNwp83t3bgGuBPzCzNkiHHOB9wLHpnCgrgoy7Pwl0B12H5BeNUBIRCYa7n3T3vZntfuAgsDRz+H8DfwT4dM4Vms6+ZnYvcC+kf0sWeSc0QklEZM7Vm9nzE/Z3ufuuC99kZk3AZuBZM7sTOOHuL0/3KeChCTKZi98FcKUtmVZKE7kYjVASEZmcj4yS7Oh6p6fpcvetk73BzCqAR4D7SDc3/TfSzUrTlhVNSyLzSSOURESCZ2bFpEPMt9z9UaAFWAW8bGZHgGXAXjNbNNl5QnNHRmQ2TNWkpBFKIiJzz9LtRn8HHHT3vwBw931Aw4T3HAG2uvukt4ay4o6MmT0MPAO0mlm7mX066Jok90ynSUkhRkRkXmwHfhd4j5m9lFk+MJMTZcUdGXe/O+gaJHftZh8P8Pikd2LUpCQiMn/c/Slg0t687t40nXNlRZARmSvTeZo1qElJRCSssqJpSWSuTDVXDKhJSUQkzBRkJKdN1pwEalISEQk7NS1JztrNPoxLTw2pSe9ERMJPQUZy0m72cR+7LxpiDPg/fEgBRkQkB6hpSXLOWAff5CXuxTgoxIiI5AgFGck5U3Xw1bO6RERyh4KM5JSpZu5V514RkdyiICM5Y6qZewsxzRcjIpJjFGQkZ3yJn0z6MMivcZdCjIhIjlGQkZywm330Ervkcd2JERHJTQoykhO+xE8ueUwz94qI5C4FGQm9qe7GqHOviEjuUpCR0HuAxy95rIao7saIiOQwBRkJtamGW/8J75/HakREZL4pyEhoTTXcWndjRERyn4KMhNZUw611N0ZEJPcpyEgoabi1iIiAgoyElIZbi4gIKMhICGm4tYiIjFGQkdDRcGsRERmjICOho+HWIiIyRkFGQmU3+y55rADT3RgRkTyjICOhMlmzUgqfx0pERCQbKMhIqEzWrLSU6nmsREREsoGCjISKTXJMo5VERPKPgoyExm72Tdp4pP4xIiL5R0FGQmOy5yqpWUlEJD8pyEgo7GbfJZ+rBGpWEhHJVwoyEgqTjVYCNSuJiOQrBRkJhclGK9UQncdKREQkmyjISChMNlpJs/mKiOQvBRkJBQeqqqp45JFHqKqqOu+YmpVERPKXgoyExo4dO7jrrru44447gi5FRESyhIKMhMbOnTvPW4uIiCjISChUVVVx3XXXAXD99ddTWVkZcEUiIpINFGQkFO7e8RESiQQAiUSCHTt2ABqxJCKS77IiyJjZ7Wb2mpkdMrMvBF2PZJ8v7rxvvJNvVVUVO3fupJhCjVgSEQkpM3vQzDrNbP8Fr/8nM3vVzA6Y2Z9PdZ6iuStxesysEPgr4L1AO/BrM/uhu78SbGUy32q+91GiH2676DEfHj1v/4bt2xlOXXym39gjr9Dz0e/Nen0iIjKrHgL+Evj7sRfM7BbgTuBKdx82s4apTpINd2SuBg65+1vungC+TfoiJM/03f84iRdPkhpIvO2YlZyfuSMlJW97T2ogQWLvSfrun3wWYBERCZ67Pwl0X/DyfwAecPfhzHs6pzpPNgSZpcDxCfvtmdckzyQPddO17Rv0f/kJUkMj+GhqWp/z0RSpoRH6v/wEXdt2kTx04b8LEREJibXAjWb2rJn90sy2TfWBwJuWpsvM7gXuBT3pOKelnMGv7SH+z69T8+2PULS2joKKyKXfPpBg9PUuej72fZJv9sxjoSIiuaugIkr0po3v7CQPUW9mz094ZZe775riU0VALXAtsA34rpk1u7tfstZ3VuWsOAEsn7C/LPPaedx9l7tvdfetdZTNW3ESjPTdmV0MPPAUPnTxvjA+NMLAA0/Rte0bCjEiItmna+x7O7NMFWIg3SrzqKc9B6SA+sk+kA1B5tfAGjNbZWYR4OPADwOuSbKBQ/LYuUs2MfloiuTR3vTzC0REJBf8I3ALgJmtBSJA12QfCDzIuPso8IfAY8BB4LvufiDYqiRblO3cREFVumOvp5zUYAJPpZNLQVUJZTs3B1meiIjMkJk9DDwDtJpZu5l9GngQaM4Myf428HuTNStBlvSRcfcfAz8Oug7JLlZVQuS6dKujD4+S6olz7r5/ofprt1NQU4qVFBG5fjlWGcH73z7SSUREspe7332JQ5+8nPMEfkdG5FJKd7TiyRSpwQQjL3TQufGviX/3AJ0b/5qRFzoyd2dSlO5oDbpUEREJiIKMZK2ynZsoKI8w9M29dN38EN4dA8C7Y3Td/BBD39xLQVlEzUsiInksK5qWRC4m2TlI993fJ/6di3SZSjp9n32MxJ52Su9cN//FiYhIVlCQkazVe/cjU74n/p0DFw86IiKSF9S0JCIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqEVaJAxs4+a2QEzS5nZ1iBrERERkfljZp/NZID9ZvawmZXO5DxB35HZD3wIeDLgOkRERGSemNlS4D8DW919I1AIfHwm5yqazcIul7sfBDCzIMsQERGR+VcERM1sBCgDOmZykqDvyEybmd1rZs+b2fNnGQq6HBEREZlc/dj3dma5d+yAu58A/hdwDDgJnHP3n87kh8z5HRkz+zmw6CKHvujuP5juedx9F7AL4Epb4rNUnoiIiFwgdbSP/nsef6en6XL3i/Z/NbMa4E5gFdALfM/MPunu/3C5P2TOg4y73zbXP0NERERC5TbgsLufATCzR4HrgcsOMqFpWhIREZGccQy41szKLN1R9lbg4ExOFPTw67vMrB24DviRmT0WZD0iIiIy99z9WeD7wF5gH+k8smsm5wp61NJuYHeQNYiIiMj8c/evAF95p+dR05KIiIiEloKMiIiIhJaCjIiIiISWgoyIiIiEloKMiIiIhJaCjIiISMAc5yR9QZcRSoEOvxYREclXAwzzFIf5BYd4su4EZwfO0Z14gEgkEnRpoaIgIyIiMg+SpNjHSX7FYZ5a182eN/YxkhyloiTKu1ds5dYrriaZTAZdZugoyIiIiMyBFM6rdPI0h3l2Uz9Pv/YbzsUGANgw0Mx/fN9Hee8V13LN6o1EiooBiEajQZYcSgoyIiIis8Bx3qCLpznMMxxlT0W6uQiguWMpd119Czetv4obWzfTUF0bcLW5Q0FGRERkBhznEF08wxGe5gh7Kjvo6u8FYHldI+9rvZab12/hxvWbWVbbGHC1uUtBRkREZBpSOK/RyR6OsoejPFd1kjN9PQAsrVnIbeuu4YZ1m7hx3WZW1i8m/VBnmWsKMiIiIhcxRIKX6GAvx3mBdl6oOE13pqloeV0jt669mu2tV7K9dRPNDUsVXAKiICMiInnPcY7Tmw4sHOfFlefYf/xNkqn0KKLVjcv5wJrt48FlZf3igCuWMQoyIiKSdwZJ8DId7KWdvbTzYlXneDNReUmUq6Lr+OwHPsHVLRvZ1tJGbUV1wBXLpSjIiIhITkuS4g26eJF2XuIEv1nex4H2t0h5Ckjfbbmt5Wq2tmxgW3MbG5Y1U1Sor8ew0J+UiIjkjLGp/l+mgxc5wb51Q7x4+FUGhmMAVJdVsKVqPf/ljk+xrbmNLc3rdbcl5BRkREQklFI4R+nhACfZx0kObkjw8tHXx+duKSos5Ir4Gu7efjtbVq1na3MbLY3LKCjQYwZziYKMiIhkvQRJ3uAMBzjFfk7y6to4+44doj8+BEBxYRHr+1fxgc03cOXKtWxauZYrVqymtLgk4MplrinIiIhIVuklxiuc5hVO8QqnObhikNc6jpAYHQGgLFLKxmQLH7vufbxr5VretWINbUtXUVKshy3mIwUZEREJRJIUR+nhFU5xkNMcpJNX685x7Oyp8fcsrKrhisrV3HLbR7hyxRretXINLY3LKCwoDLByySYKMiIiMqcc5xT9vM4Z3uAMr9HJ66viHDxxmKFEHIDCgkLWLFrOtmUbuOeWO7li+WquWLGaxuq6gKuXbKcgIyIisyKF08E5XucMh+jiDc7wZkuC1zqOjj/1GaC2opoNJc38/s13sGFZCxtXtLBuSZP6s8iMKMiIiMhlSZDkCN0coos36eINunhzZZw3Th4bv8MCUFdRzfqiVXzk2ttYv6SJdUtXsW5JEwuragKsXnKNgoyIiFxUN0O8SRdvcnZ8/VbjEIfPdIxP3Q/pByaurVjJ7930QdYuWcm6JU20Ll5JXeWCAKuXfKEgIyKSx2KMcJQe3uLs+HKkZYQ3Th2nZ7Bv/H2RomJaGpfRtqiZf7vt3bQuXsmaxStZ3bicymhZgFcg+U5BRkQkx/UR5xg9HKGHo3RzjB7a1xtvdbbT3t2Ju4+/t7G6ljVFK7hz682sXbySNYuWs3rRClbWL9JIIclKCjIiIiE3SoqT9HGMHo7Sw3F6OEYv7auSHD7TQXdmptsxdRXVrBpeyvVrr6S5YSmrF61g9aJlNDcsoypaHtBViMyMgoyISJZL4XQyQDu9HKOH4/RynF461htHz5zkRE8no8nf9lkpKixkWW0jK0sXs2PLTaxqWMqqhUtY1bCUpoVLFFYkpyjIiIgEbIQkp+jnBOdop5cTnOM4vZxqK+RY10nauzvHZ7Ud01hdy4rEYra1bODD9bfStHAJTQsX07RwCUtrFurpzZI39DddRGQOpXC6GeIUfXTQRwfn6KCPE5zj9Grj+NlTnOw9S8pT532usbqW5fFFbGpq5Y4tN7G8bhFN9YtZUb+IFfWLiUY054oIKMiIiMxYjBFO089p+jlFP6foy6z76VpTSEfPGU72dr3tbkpxYRFLaxtYXtTIzW1bWFbbyPK6RpbVNbK8tpHl9Y2aHE5kmhRkREQmcJxBEnQywGn66WSAMxO2u9YXc6q3i1O9Z8+brXZMWaSUxTX1LC6o5+rVG1hSs5ClNQ0sqVnIkpp6ltU1srCyhoKCggCuTiT3KMiISF6IMcIZBjjDIGcZzGynl04G6V5dSGdfD53nuhkcjr3t85GiYhZV17EoUUfrkiZubtvCogX1LF5QR2N1HUtqFrK4pp7qaAVmFsAViuQnBRkRCaUESc4ySA9DnGWIrkxASS9D6WMtRXT29dDV18PARcIJQE15FY3VtTQU17K1eT0NVbU0VtexKBNQGqvT+zXllQooIlkolEHmMN18jh9QQ5QaoiwgygLKJmynlzKKMfQfj0g2G2vKOUecXmL0EqOHGL0MZdbp/W6G6G0u4uzAOc7299IXG7zo+QoLCqmvrGZhZQ11kQq2tbTRUFVLfeUCGqpqaaiuob6yhobqGhqqaokUFc/zFYvIbAplkBmNwNMVHXQP9J33gLILFRUWUlNeRU15JQvKKil/c5hqolRTygJKqSZKFaVUUUp1Zj22VFJCEWrDFpmOUVL0E+cccfoZpi+znV7H6CNOb+a1c8QYaCmhd7Cf3qF+egb7zpsD5UJlkVJqK6qoraimtrSMpoVLqK+spr5yAbUVC6ivrKaucgENVTXUVy5gQVml+p+I5JFAg4yZfRW4A0gAbwI73b13qs+1Lm7il1/5BgDxkWG6B/roGeyje6CP3sH0f4w9mXXvUD89A5n/MFeOcnjoLL1D/ZwbGjhvWu6LKS+JUhUtTy9lFZS9GaeSkvGl4oJ1OSVUEBlfV1BCORFKwpkXJcc5zjBJhkgwwDADJBicsO5nmIEL1mPLYHMJfbEB+mND9MUGL9qnZCIzo7qsggVl6V8qFpREWVrTQE1FJTXlVSwoq6SmPLNdXpkOLuXV1FRUavSOSI4ys9uBrwOFwDfd/YGZnCfob9ifAfe7+6iZ/RlwP/BfL+cEpcUlmdEACy/rB6dSKfrig/QO9tMXG6QvNsi5TMDpGxrkXGyAvqEBzmWO9cUG6FuVoiM2wEC8k/7Y4CXb3C9UXFhEeWmUipIo5SVRykujlB6OU06EMiJEKaaMYsqIjK+jE9YXLqUUUTphrTtHuWuUFHFGiDM6vo4x8rZliMT4emjCeuy1QRLEV5UyGI8xOBxjYDjGYDzGSHJ0WnVUlESpjJZTGS2jKlpBZbSMpbUNVEXLqc4E/epoBdVlFVSVlVNdVkl1dGy7gqrSct0lEZFxZlYI/BXwXqAd+LWZ/dDdX7nccwUaZNz9pxN29wAfma+fXVBQMP7b4UwlU0kG4jH644P0x4bojw3SHx9iID7EQDzGQHxofH8w88Ux9gUytDbCqXiMoeFehobj6ePDcZKpS99iv5SiwkKixSWURkqIFpdQUhyhtDhCSXGEorcGKaFofIlQmFmKKMmsx14rziwRiiimYHy/KLM9cV00vv7tUji+2HnbBRNeMyzzmmX25r4Pk+OkMksSx3GSpEjipDLr9P7526PnLcnx7ZHM9tg6QZJRkoyQIsEoIyQZIclwZp1glOHMOkGS4QnrYUYZbS5neCRBfCTB8EiC2Mgw8cQwsZHhSZtcLqWwoJCKkihlJaWZJUplaRV1JVFW1C2mvKSU8tJ0qK4oLaOytIyK0jIqSn+7PxZaKkvLqSiN6mGBIjLbrgYOuftbAGb2beBOIFxB5gL3AN8JuojLUVhQSHVZ+rfQ2eDuJEZHGByOM5SIEU8kGErEiSXiDA6n17Hh4fO+6GKJ9DK2nxgdIT6SIJ4YTn8xri1mYCRB9+gI8ZEYw6MJEiMjDI+OMDySIJEcmdGX5WwqsALMjAIzzNLhxozxESJjYWfiiJGxZkHHx/fdM6HFU+lt97fNljrfigoLiRQWU1IcoaSomEhxMSVF6aAZKYpQWlxBWXGEaHXteACNFBWPB9OySCmlkch5QbWspJRopJSykpL0OvOeskiU8pJSIkXFGl0jItluKXB8wn47cM1MTjTnQcbMfg4susihL7r7DzLv+SIwCnxrkvPcC9yb2R2uvuem/bNda4jUA11BFzFbUp4Ch2nGqVBd+2gyyWgyOWmn9MsUquufA7p+XX9uX/89lzzSOo9V8BtOPraMP65/h6cpNbPnJ+zvcvdd7/CcbzPnQcbdb5vsuJn9PvBB4FafpPdt5uJ3ZT7zvLtvnc06wySfrz+frx10/bp+XX++Xv8FgWDOufvtc/wjTgDLJ+wvy7x22QLtfZfpsfxHwA53HwqyFhEREZk3vwbWmNkqM4sAHwd+OJMTBd1H5i+BEuBnmTb9Pe7+74MtSUREROZSZrTyHwKPkR5+/aC7H5jJuYIetbR6hh+d9Ta2kMnn68/nawddv64/v+Xz9efctbv7j4Efv9Pz2FSTwomIiIhkK81QJSIiIqEV2iBjZl81s1fN7DdmttvMFgRd03wxs4+a2QEzS5lZ3vTgN7Pbzew1MztkZl8Iup75ZGYPmlmnmeXltANmttzMnjCzVzJ/9z8TdE3zxcxKzew5M3s5c+1/HHRNQTCzQjN70cz+Oeha5puZHTGzfWb20nyPXgqD0AYZ0o832Oju7wJeJ/14g3yxH/gQ8GTQhcyXCdNZvx9oA+42s7Zgq5pXDwFzPRwym40Cn3f3NuBa4A/y6M9/GHiPu18JbAJuN7NrA64pCJ8BDgZdRIBucfdN+Tr8fDKhDTLu/lN3H3tQzB7SY9DzgrsfdPfXgq5jno1PZ+3uCWBsOuu84O5PAt1B1xEUdz/p7nsz2/2kv9CWBlvV/PC0gcxucWbJq86NZrYM+DfAN4OuRbJPaIPMBe4BfhJ0ETKnLjaddV58kcn5zKwJ2Aw8G2wl8yfTrPIS0An8zN3z5tozvkZ6zrFgnzkSHAd+amYvZGa5lwmCnkdmUrP1eIMwms61i+QbM6sAHgHuc/e+oOuZL+6eBDZl+gLuNrON7p4X/aXM7INAp7u/YGbvDrqegNzg7ifMrIH0vGuvZu7SClkeZGbr8QZhNNW156FZm85awsnMikmHmG+5+6NB1xMEd+81sydI95fKiyADbAd2mNkHgFKgysz+wd0/GXBd88bdT2TWnWa2m3RTu4JMRmiblvR4g7wza9NZS/hYeurvvwMOuvtfBF3PfDKzhWOjMs0sCrwXeDXYquaPu9/v7svcvYn0v/t/zacQY2blZlY5tg28j/wJsdMS2iBD+vEGlaRvs71kZn8TdEHzxczuMrN24DrgR2b2WNA1zbVMx+6x6awPAt+d6XTWYWRmDwPPAK1m1m5mnw66pnm2Hfhd4D2Zf+8vZX5DzweLgSfM7DekA/3P3D3vhiDnsUbgKTN7GXgO+JG7/0vANWUVzewrIiIioRXmOzIiIiKS5xRkREREJLQUZERERCS0FGREREQktBRkREREJLQUZERERCS0FGREREQktBRkROSymNmDZtZpZppdVEQCpyAjIpfrIdLP+hERCZyCjIhclJk9YWbvzWz/qZn9X4DMU3e7Ay1ORCQjq59+LSKB+grwJ2bWAGwGdgRcj4jI2yjIiMhFufuTmadOfw54t7sng65JRORCaloSkYsysytIP3k54e79QdcjInIxCjIi8jZmthj4FnAnMGBm6twrIllJQUZEzmNmZcCjwOfd/SDwP0n3lxk7/jDwDNBqZu1m9ulgKhURAXP3oGsQERERmRHdkREREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0FKQERERkdBSkBEREZHQUpARERGR0Pr/xDElau02SmMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "w = paddle.to_tensor([0.2, 2])\n",
    "model = OptimizedFunction(w)\n",
    "opt = RMSprop(init_lr=0.1, model=model, beta=0.9, epsilon=1e-7)\n",
    "train_and_plot_f(model, opt, epoch=50, fig_name='opti-vis-para3.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**简单拟合实验**  训练单层线性网络，进行简单的拟合实验。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAEICAYAAAB74HFBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcXGd95/vP99TSuxZLbVmyJcsbNrbBAost8GKxcWIcwCQ3w8BlS8Jg7k1I4GZhSyYxuYTATICQIQNjwMEJewwMhiG5GDDxdQLGkhd5A7zKlixZsmRtvdbymz/Oaaktd7tb3VVdp6q/79erXnW2Oud3qquf+tXzPOc5igjMzMzMbG6SVgdgZmZm1s6cTJmZmZnNg5MpMzMzs3lwMmVmZmY2D06mzMzMzObByZSZmZnZPDiZ6jCS7pT00hYef52kQ5IKTdh3SDq90fs1MzuapPVZmVOc5fafl/TBZsdl+eRkqsNExDkR8SMASZdL+kIzjyfpQUkvn3T8hyKiPyJqzTyumZlZXjiZsmnN9heZmZnZYuZkqsNM1BRJuhh4P/Afs2a327L1SyV9TtIOSdslfXCiSU7Sb0r6N0kfl7QHuFzSaZJ+KGmPpMckfVHSsmz7fwTWAd/OjvHuo6vGJa2RdI2kvZLulfS2SbFeLulrkv5B0sGsiXLjLM9zafa63ZK2SvpTSUm27nRJ/yppfxbzV7Plys5tl6QDkm6XdG7D3nwza5qsLPl69j//gKTfn7TucklXS/pqVpbcLOm8SeufLulHkvZl5cyrJ63rkfTRrBzZL+kGST2TDv0GSQ9lZcmfHEO8b8vKvL1ZGbgmWz5tOSTpEkl3ZeewXdIfzetNswXjZKpDRcS/AB8Cvpo1u00ULJ8HqsDpwLOAXwb+06SXPg+4H1gF/CUg4K+ANcDTgbXA5dkx3gQ8BLwqO8Z/mSKUrwDbstf/BvAhSRdMWv/qbJtlwDXAJ2d5iv8NWAqcCrwEeDPwW9m6/xf4HrAcOCnbluxcXww8LXvta4E9szyembVI9kPp28BtwInAhcC7JP3KpM0uBf4JOA74EvA/JZUklbLXfg84Hvg94IuSzsxe99fA+cAvZa99N1CftN8XAWdmx/wzSU+fRbwXkJabrwVWA1tJyzl46nLoc8DbI2IAOBf44UzHsnxwMrWISFoFXAK8KyKGImIX8HHgdZM2eyQi/ltEVCNiJCLujYhrI2IsInYDHyNNXmZzvLXAC4H3RMRoRNwKfJY08ZlwQ0R8N+tj9Y/AeVPs6uj9FrKY3xcRByPiQeCjwJuyTSrAycCa7Lg3TFo+AJwFKCLujogdszkXM2up5wCDEfEXETEeEfcDn+GJZdfmiLg6Iiqk5VQ38Pzs0Q98OHvtD4HvAK/PkrTfBt4ZEdsjohYR/x4RY5P2+4GsLLyNNJmbsYwC3gBcGRE3Z/t6H/ACSet56nKoApwtaUlEPB4RNx/zO2Ut4WRqcTkZKAE7surufcD/IP21NuHhyS+QtErSV7Iq5wPAF4CVszzeGmBvRByctGwr6S/LCTsnTQ8D3bPoq7UyO4+t0+z33aQ1aj/NqvR/GyArRD8J/B2wS9IVkpbM8lzMrHVOBtZMlFtZ2fV+0hr0CYfLroioc6RGfA3wcLZswkR5sZI06brvKY59dBnVP4t41zCpfIqIQ6S1TyfOUA79H6Q/eLdmXRVeMItjWQ44mepscdT8w8AYsDIilmWPJRFxzlO85kPZsmdExBLgjaSJynTbT/YIcJykgUnL1gHbj+UkpvAYR2qfnrTfiNgZEW+LiDXA24H/rmxIhYj424g4HzibtJr9j+cZi5k138PAA5PKrWURMRARl0zaZu3ERFbjdBJpGfQIsHaiT2Vmorx4DBgFTmtwvI8wqXyS1Aes4EgZNWU5FBE3RcSlpD9w/yfwtQbHZU3iZKqzPQqsnyhEsqrk7wEflbREUqK0g/lTNdsNAIeA/ZJO5MnJx6Ok/ZaeJCIeBv4d+CtJ3ZKeCbyVtHZrzrImwa8BfylpQNLJwB9M7FfSf5B0Urb546QJX13ScyQ9L+tDMURaiNaffAQzy5mfAgclvSfrMF6QdK6k50za5nxJv57VbL+L9IfjT4AbSWuU3p31oXop8CrgK1lt1ZXAx7IO7gVJL5DUNc94vwz8lqQN2b4+BNwYEQ9OVw5JKkt6g6SlWVPlAVw+tQ0nU53tn7LnPZIm2t7fDJSBu0gTjatJO0hO5wPAs4H9wP8CvnHU+r8C/jSrep/qypPXA+tJf6l9E/jziPj+sZ/Kk/weaUF0P3ADaYfTK7N1zwFulHSItFP7O7M+FktI+1k8TloFvwf4rw2IxcyaKPsB9UpgA/AAaY3SZ0k7cE/4FvAfSf+/3wT8ekRUImKcNHl6Rfa6/w68OSJ+lr3uj4DbgZuAvcBHmOd3Y1bG/Wfg68AO0pqvif5dT1UOvQl4MOtS8X+R9r2yNqCIp2qlMTMzyzdJlwOnR8QbWx2LLU6umTIzMzObBydTZmZmZvPgZj4zMzOzeXDNlJmZmdk8LOiNbFeuXBnr169fyEOaWYtt3rz5sYgYbHUc8+Xyy2zxmW35taDJ1Pr169m0adNCHtLMWkzS1pm3yj+XX2aLz2zLLzfzmZmZmc2DkykzMzOzeXAyZWZmZjYPMyZT2T3VfirpNkl3SvpAtvzzkh6QdGv22ND8cM3MzMzyZTYd0MeACyLiUHZjxhsk/XO27o8j4urmhWdmZmaWbzMmU5GO6nkomy1lD4/0aWZmZsYs+0xJKki6FdgFXBsRN2ar/lLSFkkfl9Q1zWsvk7RJ0qbdu3c3KGwzMzOzfJhVMhURtYjYAJwEPFfSucD7gLOA5wDHAe+Z5rVXRMTGiNg4ODi7cfuq9+7hwH/+IdWH9s1qezOzPBm7/WEqD+1pdRhmtkCO6Wq+iNgHXAdcHBE7IjUG/D3w3EYFVX1gHwc/eD21hw80apdmZgtm/K5HqG59rNVhmNkCmc3VfIOSlmXTPcBFwM8krc6WCXgNcEfDolL2XHfXLDObH0lXStol6UlllKQ/lBSSVjbymElfF/Xh8Ubu0sxybDY1U6uB6yRtAW4i7TP1HeCLkm4HbgdWAh9sWFTKsqlwMmVm8/Z54OKjF0paC/wy8FCjD6jeMjE01ujdmllOzeZqvi3As6ZYfkFTIgKUTCRTzTqCmS0WEXG9pPVTrPo48G7gW40+ZtLbRfUR9/k0WyzyOQK6m/nMrIkkXQpsj4jbZthuTlcjq7cMlRpRqc03VDNrAzlNptzMZ2bNIakXeD/wZzNtO5erkSFLpoD6sJv6zBaDfCZTWTOfcykza4LTgFOA2yQ9SDrky82STmjUAZLedNi9cCd0s0VhNreTWXhu5jOzJomI24HjJ+azhGpjRDRsLAP1TdRMOZkyWwzyWTPlZj4zaxBJXwZ+DJwpaZuktzb7mIdrpnxFn9mikM+aKV/NZ2YNEhGvn2H9+kYfU6UClApu5jNbJHJZMyU385lZm0t6y+6AbrZI5DKZcjOfmbU79XW5ZspskchnMuVmPjNrc0lPmbr7TJktCvlMpiZyKTfzmVmbUl8XMTJOuIbdrOPlNJlyM5+ZtbektwwBMVJpdShm1mT5TKbczGdmbW5iFPRwJ3SzjpfPZMpX85lZm5sYa8oDd5p1vlwmU3Izn5m1uYlR0F0zZdb5cplMuZnPzNqdussgqA+5Zsqs0+UzmXIzn5m1OSVCPWWPNWW2COQ0mUqzKV9SbGbtTB4F3WxRyGcy5WY+M+sASa9HQTdbDPKZTLmZz8w6QFoz5WTKrNPlNJny1Xxm1v6Svi4YrxLVWqtDMbMmmjGZktQt6aeSbpN0p6QPZMtPkXSjpHslfVVSuVFBaaKZzzVTZtbGJgbu9BV9Zp1tNjVTY8AFEXEesAG4WNLzgY8AH4+I04HHgbc2LKqJZj7nUmbWxhKPgm62KMyYTEXqUDZbyh4BXABcnS2/CnhN46JyM5+ZtT9lo6C7E7pZZ5tVnylJBUm3AruAa4H7gH0RUc022QacOM1rL5O0SdKm3bt3zy6qiaER3MxnZm0syUZB9/AIZp1tVslURNQiYgNwEvBc4KzZHiAiroiIjRGxcXBwcHYvcjOfmXUAlYpQLLhmyqzDHdPVfBGxD7gOeAGwTFIxW3USsL1xUbmZz8w6Q9Ln4RHMOt1sruYblLQsm+4BLgLuJk2qfiPb7C3AtxoWlXw1n5k1hqQrJe2SdMekZf9V0s8kbZH0zYkyrinH7y0TQ27mM+tks6mZWg1cJ2kLcBNwbUR8B3gP8AeS7gVWAJ9rVFByM5+ZNc7ngYuPWnYtcG5EPBP4BfC+Zh086e1yzZRZhyvOtEFEbAGeNcXy+0n7TzWem/nMrEEi4npJ649a9r1Jsz/hSC17w6k3vdlxRKDDvxTNrJPkewR0N/OZWfP9NvDPU62Y09XIR0l6uyCCGK3MJ0Yzy7GcJlPpkyumzKyZJP0JUAW+ONX6OV2NfPQxsuERwqOgm3WsGZv5WsLNfGbWZJJ+E3glcGFE8wqbJBu4sz48RoH+Zh3GzFoon8mUm/nMrIkkXQy8G3hJRAw39ViHbynjmimzTpXrZj5fzWdm8yXpy8CPgTMlbZP0VuCTwABwraRbJX26acfvKYM8CrpZJ8tlzZTczGdmDRIRr59iccOGcpmJEqHusmumzDpYTmum3MxnZp1DHgXdrKPlNJnKnp1LmVkHSHq7PAq6WQfLZzKVNfM18QIbM7MFo17XTJl1snwmU27mM7MOkvSWYbxKVGutDsXMmiCnyVT27FzKzDqAsrGm3AndrDPlM5ny1Xxm1kGSbKwpN/WZdaZcJlNyM5+ZdZDDt5TxWFNmHSmXyZSb+cyskxy+pYzvz2fWkfKZTLmZz8w6SakAxcQ1U2YdKp/JVNbMF27mM7MOIImkt8t9psw6VE6TqezZuZSZdQj1+pYyZp0qn8lU4g7oZtZZkr4u3+zYrEPlM5mS+0yZWWeZqJnynR3MOs+MyZSktZKuk3SXpDslvTNbfrmk7ZJuzR6XNCqow0MjuMwxsw6h3jLUgxirtDoUM2uw4iy2qQJ/GBE3SxoANku6Nlv38Yj466ZEJtzMZ2YdY2J4hBgah+5yi6Mxs0aaMZmKiB3Ajmz6oKS7gRObHRiSm/nMrGNo0ijohRUtDsbMGuqY+kxJWg88C7gxW/QOSVskXSlp+TSvuUzSJkmbdu/efQyRybmUmXWMwzVT7oRu1nFmnUxJ6ge+DrwrIg4AnwJOAzaQ1lx9dKrXRcQVEbExIjYODg7OPjI385lZB1FvCfD9+cw60aySKUkl0kTqixHxDYCIeDQiahFRBz4DPLehkbmZz8w6iJIE9ZSIIddMmXWa2VzNJ+BzwN0R8bFJy1dP2uzXgDsaG5l8NZ+ZdRR5FHSzjjSbq/leCLwJuF3Srdmy9wOvl7SBNOV5EHh7IwOTm/nMrAEkXQm8EtgVEedmy44DvgqsJy2/XhsRjzc7lqS3TP3QaLMPY2YLbDZX893AkRu8TPbdxocziZv5zKwxPg98EviHScveC/wgIj4s6b3Z/HuaHYh6y8SuA80+jJktsHyOgA5u5jOzhoiI64G9Ry2+FLgqm74KeM1CxJL0dhFjVaJaX4jDmdkCyW8yJQg385lZc6zKxtAD2AmsmmqjOQ/tMg31pWNNxYg7oZt1khwnU27mM7Pmi/RmeVMWNnMe2mUaE2NN1YfcCd2sk+Q3mXIzn5k1z6MTVyRnz7sW4qATo6B74E6zzpLfZMpX85lZ81wDvCWbfgvwrYU46OGaKQ+PYNZRcpxMuZnPzOZP0peBHwNnStom6a3Ah4GLJN0DvDybb75yAYoJ4WTKrKPMZpyplpCb+cysASLi9dOsunBBAwEkkfR2Ufco6GYdJcc1U7iZz8w6jnrLxIhrpsw6SY6TKRFu5jOzDqPesq/mM+sw+U2m3MxnZh0o6e0iRsb8Y9Gsg+Q3mXIzn5l1IPWWoRbEWLXVoZhZg+Q4mfLVfGbWeZI+jzVl1mlym0z5aj4z60TyKOhmHSe3yZSb+cysEyUeBd2s4+Q4mXIzn5l1nolbyngUdLPOkd9kKhHhmikz6zBKEtTfTX3fcKtDMbMGyW8yJdxnysw6UmFlP7U9h1odhpk1SH6TqcTNfGbWmQor+omDo8RYpdWhmFkD5DeZktwB3cw6UmFFPwC1PUMtjsTMGiHHyRRu5jOzjpQcTqbc1GfWCWZMpiStlXSdpLsk3Snpndny4yRdK+me7Hl5IwOTm/nMrEMl3SXU3+VkyqxDzKZmqgr8YUScDTwf+F1JZwPvBX4QEWcAP8jmG8fNfGbWwQrH9VN3MmXWEWZMpiJiR0TcnE0fBO4GTgQuBa7KNrsKeE1DI3PFlJl1sMLKfuoHRohx36PPrN0dU58pSeuBZwE3AqsiYke2aiewaprXXCZpk6RNu3fvPobIRH3nIer7Ro4lRDOztnC4E/pe106ZtbtZJ1OS+oGvA++KiAOT10VEME138Yi4IiI2RsTGwcHB2UcmMX7DQ+xc/zezf42ZWZs43An9MSdTZu1uVsmUpBJpIvXFiPhGtvhRSauz9auBXQ2NTOlT7Pf9q8ys8yQ9ZdRbdid0sw4wm6v5BHwOuDsiPjZp1TXAW7LptwDfamRgStTI3ZmZ5U5h5YA7oZt1gNnUTL0QeBNwgaRbs8clwIeBiyTdA7w8m28cOZkys+aS9P9kQ77cIenLkroX8viFFX3U948QFXdCN2tnxZk2iIgbONzo9iQXNjacScet1Jq1azMzJJ0I/D5wdkSMSPoa8Drg8wsVQ7JiAEhHQi+esHShDmtmDZbbEdBjyPesMrOmKwI9kopAL/DIQh684JHQzTpCfpOpESdTZtY8EbEd+GvgIWAHsD8ivjd5mzkP7TJLSW8Z9ZTdb8qszeU2mVJ/udUhmFkHy26BdSlwCrAG6JP0xsnbzHlol2NQWNnvmimzNpfbZCpZ2dvqEMyss70ceCAidkdEBfgG8EsLHUSyop/6/mH3EzVrY7lNpgqDfa0Owcw620PA8yX1ZkPAXEh6u6wFVVjRDwG1vUMLfWgza5DcJlM9rz2n1SGYWQeLiBuBq4GbgdtJy8MrFjqOiU7o9T0HF/rQZtYgMw6N0Cq9bzqPob+/hfpO9yUws+aIiD8H/ryVMai3jHpK7jdl1sZyWzMFUDihn6jWWx2GmVnTSKJwnDuhm7WzXCdTFBNwMmVmHS5Z2U993zBRdSd0s3aU62RKTqbMbBFwJ3Sz9pbrZIpi4mY+M+t4Rzqhu6nPrB3lOplyzZSZLQbq60JdRfebMmtTuU6mXDNlZouBJJKVA06mzNpUvpOpgmumzGxxKKzop/74sH9AmrWhXCdTbuYzs8Ui7YQe1B93J3SzdpPrZIpiQtSi1VGYmTXdRCd0N/WZtZ9cJ1OumTKzxUL9XVB2J3SzdpTrZIpiAvUg6k6ozKyzSaKw0iOhm7WjXCdTKqbhPVL4Cx5/2zUtjsbMrLnSTuhDRM0/IM3ayYzJlKQrJe2SdMekZZdL2i7p1uxxSVOiKx4Jb/izNzflEGZmeVFY0Q91d0I3azezqZn6PHDxFMs/HhEbssd3GxtWppjrijMzs4ZyJ3Sz9jRjthIR1wN7FyCWJ1FBrTismVlLaKDbndDN2tB8qn7eIWlL1gy4fLqNJF0maZOkTbt37z7G6JxMmdniIYnCij4nU2ZtZq7J1KeA04ANwA7go9NtGBFXRMTGiNg4ODh4TAeJococwzMza0+FFf3U9w75KmazNjKnZCoiHo2IWkTUgc8Az21sWKn6/tFm7NbMLLeOdEIfbnUoZjZLc0qmJK2eNPtrwB3TbTsfsX+sGbs1M8utZMUA4E7oZu2kONMGkr4MvBRYKWkb8OfASyVtAAJ4EHh7M4KrH3AyZWaLS7KkG/WUqG7bS/lpJ7Q6HDObhRmTqYh4/RSLP9eEWJ6k+xWnM/Kl2xfiUGa2CElaBnwWOJf0x+FvR8SPWxwTpfWDjP9iBzFeReUZi2kza7FcD+TU84ZnUjxrZavDMLPO9QngXyLiLOA84O4WxwNA8bRBqAWVrXtaHYqZzUKukylJqK90eH70B/e3MBoz6ySSlgIvJqtpj4jxiNjX2qhShZUDaKCbyv27Wh2Kmc1CrpMpAHRkrKk9L/+HFgZiZh3mFGA38PeSbpH0WUl9kzeY1zh58yCJ0qmD1Hbsoz48vmDHNbO5aYNkqtUBmFmHKgLPBj4VEc8ChoD3Tt5gPuPkzVfp1OMhoPLAwiVxZjY3+U+mzMyaYxuwLSJuzOavJk2ucqGwrJfkuD4q9zuZMsu7/CdTctWUmTVeROwEHpZ0ZrboQuCuFob0JKVTj6f+2EHqB0ZaHYqZPYU2SKZaHYCZdbDfA74oaQvp7bE+1OJ4nqB0atq06Nops3zL/wAmrpkysyaJiFuBja2OYzpJXxeFE5ZSuX8X5fPWIpeHZrmU+5oplx1mtpiVTh2kvn+E+t6hVodiZtPIfzI10NXqEMzMWqZ48kpI5DGnzHIs98nU8qte0+oQzMxaJukuUTxxOZX7dxMRrQ7HzKaQ+2SqcMIApWevbnUYZmYtUzp1kBgep/bo/laHYmZTyH0yBRCV2pHp8WoLIzEzW3jFdSugmFC5z1f1meVRWyRT5Uk1U3HIt1Yws8VFxQKlk1dSefAxolZvdThmdpS2SKaWfeqV9L4tHZi4PlRpcTRmZguveOogjFepbn+81aGY2VHaIplST4mul50CQAy5ZsrMFp/immWoq+ir+sxyqC2SKQD1lQAI10yZ2SKkJKF4yiDVh/YSFfcdNcuTtkmmkv4y4D5TZrZ4lU4dhFqdykN7Wx2KmU3SNsmU+rJkys18ZrZIFY5fgvq7qN7npj6zPGmjZMrNfGa2uEmidMog1Ucepz7qH5ZmeTFjMiXpSkm7JN0xadlxkq6VdE/2vLy5YR6pmaq7ZsrMFrHSacdDQOWeR1sdipllZlMz9Xng4qOWvRf4QUScAfwgm28quc+UmRmF5X0UT1zO2G0PUx9xeWiWBzMmUxFxPXB0b8dLgauy6auApt9Az818ZmaprueeCtU6YzdvbXUoZsbc+0ytiogd2fROYNV0G0q6TNImSZt27577rRDUUwK5A7qZWWFZL+Wz11D5xU5qjx1sdThmi968O6BHehvzaW9lHhFXRMTGiNg4ODg45+NIQr0l10yZmQFdG9ah7hKjN95PWgybWavMNZl6VNJqgOx5Qa7TVX+Z+tA4Q5/dTO2RAwtxSDOzXFK5SNf566ntOkD1Ad8A2ayV5ppMXQO8JZt+C/CtxoTz1NRXZvh/bGbf277Nnl/90kIc0swst0qnryJZ0c/oTQ8QlVqrwzFbtGYzNMKXgR8DZ0raJumtwIeBiyTdA7w8m2++SVXZtW2umTKzxU2J6H7eqcTwOGO3P9zqcMwWreJMG0TE66dZdWGDY5lZpX54Mmr1p9jQzGx2JBWATcD2iHhlq+M5VsVVSymeOsj4Hdson3ECyUB3q0MyW3TaZgR0gKhOSqCqTqbMrCHeCdzd6iDmo3vjKSAxetP9rQ7FbFFqq2TqCQlUzVevmNn8SDoJ+FXgs62OZT6Svi66nrmW6tY9VHfsa3U4ZotOWyVTkztYupnPzBrgb4B3A1MWKI0aJ28hlM85CfV3MfqT+4i6f2yaLaS2SqZwM5+ZNYikVwK7ImLzdNs0apy8haBiQvdzTqW+b5jKz3fM/AIza5i2SqbCzXxm1jgvBF4t6UHgK8AFkr7Q2pDmp3jyCgqrlzJ681bqox7g2GyhtFUyVTihv9UhmFmHiIj3RcRJEbEeeB3ww4h4Y4vDmhdJdD/vNKjUGPnXnxF11+CbLYS2SqZWXvebrQ7BzCzXCsv76H7h6dQe2cfov9/rW82YLYAZx5nKk+Ipy1sdgpl1oIj4EfCjFofRMOUzTqB+aIzxWx8i6e+ma8O6Vodk1tHaKpkyM7PZ6dqwjjg4ytgtW1F/F+XTV7U6JLOO1VbNfEeLqu9FZWY2FUl0v/CMtEP6v91D9RGPP2XWLG2XTA3++K2ovwxAjDmZMjObjgoJvS87m2RJD8PX3UXt8aFWh2TWkdoumSo/fy1LPnhBOjNWbW0wZmY5p64ivRedgwoFhq+9k/rweKtDMus4bZdMAairALhmysxsNpL+bnovOocYqzB87R1PuJuEmc1fWyZTdKf95sM1U2Zms1JY0U/Py55O/fEhRq6727ecMWugtkym1JUlU6NOpszMZqt00nF0v+B0qtsfT2uoxjxKulkjtGkylTbz4WY+M7NjUj5zNd0vPIPazv0Mfec2avuGWx2SWdtrz2TKzXxmZnNWftoJ9F78DGK8ytB3bqXy8N5Wh2TW1toymWKimc81U2Zmc1JctZS+V20gGehm5Pt3MrblYd96xmyO2jKZOnw1n/tMmZnNWdLfTd+vnkdx/UrGNj/IyPU/92DIZnMwr9vJSHoQOAjUgGpEbGxEUDMe1818ZmYNoWKBnpeexfiWhxm7eStDB0boveBskr6uVodm1jYaUTP1sojYsFCJFBy5mm/06ruoD3kAOjOz+ZBE13nr6LnwbOr7Rhj69i1UHt7T6rDM2kZbNvORNfMNX3UbBz/0/7c4GDOzzlBat4K+V56HukqMfP8uhn9wF/VDo60Oyyz35ptMBfA9SZslXTbVBpIuk7RJ0qbdu3fP83DZPrsntU7W6g3Zp5mZQWF5H32vfhZd56+nuv1xDn1zM2O3byPqLmvNpjPfZOpFEfFs4BXA70p68dEbRMQVEbExIjYODg7O83CpZMmRtnx1Fxn93r2Ekyozs4ZQIaHrmWvp//XzKa5extimBxj61i1UH93f6tDMcmleyVREbM+edwHfBJ7biKBmkizr4fg7fwdKCcNf2MKeX/kChz7xk4U4tJnZopH0d9P78nPoufBsolJj+LtbGLnhF9RHPXK62WRzvppPUh+QRMTBbPqXgb9oWGQzKJ19PFTq1O57HIDaVv9iMjNrhtK6FWkN1W0PMX7HdqoP7aF8zomUz1pz+IIgs8VsPv8Fq4BvSprYz5ci4l8aEtUcqMf/0GZmzaJSge6Np1A6/XjGbnqAsZu3Mnb7NspyYyOcAAARRUlEQVRPX0P57DUkPeVWh2jWMnPOQCLifuC8BsYyL4c+8m8M/OmLSfo9NoqZWbMUlvXRe9G51PYcYmzLw4xveZjxO7dTftoJlM89kaS/u9Uhmi249hwaYRpDf3tjq0MwszYhaa2k6yTdJelOSe9sdUztpLCin96XPZ2+Xz+f0qmDjP9sB4eu3sTIDb+gtt83T7bFpaPaxmLct0Ews1mrAn8YETdLGgA2S7o2Iu5qdWDtpLC0l54XPY2uDesYv2M747/YSeWeRymsXkb5jFUUT16BioVWh2nWVG2dTK34/pupPbiPff/pGsDJlJnNXkTsAHZk0wcl3Q2cCDiZmoOkv5vu559G+by1VH6+g/F7djFy/c+hXKB0yiDlM04gWdlP1s/WrKO0dTLVfeGpAIeTqdr2g60Mx8zalKT1wLOAG49afhlwGcC6desWPK52lPSU6dpwMuXz1lHbuZ/KPY9SuXcXlZ/vJFnWS+mMVZROO94d1q2jdESfqZU/+k0A6tsOtDYQM2s7kvqBrwPviognFCLNGHR4sZBEcfUyel58JgOvex7dv3Q6KhUYu+kBDn31Rob+ZQvjd2337WqsI7R1zdSErpesp+eNz2T8+q2tDsXM2oikEmki9cWI+Ear4+lUKhcpn7ma8pmrqe0bonLfbqoP7WH0xvvhxvtJVvSnY1mtW0GyvNdNgdZ2OiKZAig+bQUjX9hCjFRQT6nV4ZhZzin9xv4ccHdEfKzV8SwWhWV9FM7vg/PXU9s/TPWhPVS37mHslq2M3bIVDXRTWnschTXLKZ6wFJXced3yr6OSKYDqPXsoPfOEFkdjZm3ghcCbgNsl3Zote39EfLeFMS0qhaW9FJ7RS9cz1lIfHqf6cJpYjf9sB9z1CEgUBgcorF5Kcc0yCoNLUKEjeqdYh+m4ZGrXeZ+mdP5qVl7/WyS97uBoZlOLiBsAtyflRNJbPtwUGNUatV0HqO7YT/WRfenAoLc9DIWEwqolFFcvo3D8AIUVA665slzonGTqjOMOT1c272Ds2vvpufSsFkZkZmZzoWKB4prlFNcsh/MhxqtUd+6ntmMf1Uf2Mbb5wWxDSJb3UTh+SVqDNThAsqTHfa5swXVMMnX0bWRiv68QMTPrBCoXKa1bQWld2gJRH61Q232Q2u4D1HYfpHLfLio/25Fu21UkWTlAYUU/heP6SI7rJ1nS7QTLmqpjkimAgT97CQf/4l8BqD6wr8XRmJlZMyTdJZK1x1Fam7ZIRD2o7x9OE6xdB6g9dojxR7ZBRPqCYiFLrPrSJGt5H8myXjcRWsN0VDK15AMvO5xM1R54vMXRmJnZQlAiCsv7KCzvg6elFyBFrU798WFqew9R23uI+t6hdPDQrAYLQH1dJMt6KSzrJckehaW9qKujvhptAXTcJ+aE7X/AY7/yBar3O5kyM1usVEgorOynsLL/8LKIIA6OUts7lNZk7Rumvm+Y8Z37oVY/8tqeEsmSnkmP7vR5oMe1WTaljkumCmuWUN64htFr72t1KGZmliOSUJYgTRb1IIZGDydX9f0j1A+MUN22lxipPHEfPeU0uervRgPdJP1dJP3ZfF8XStw3azHquGQKoHDKMurbD1L52W5KZ/kWEGZmNj0lQgNpzRNrVzxhXVSq1A+MUj8wkj1GqR8cobpzP3HfrqN2lDUdZg9N8ayuojvDd6COTKa6XrKeg8CuZ36KE+5/J4WTlrY6JDMza0MqFdNO6yv6n7QuanVieIz6wVHqh8aoHxolDo5SHxqjuusAMTwO9XjiiwoJ6i2T9JbTWq7eMurtSpf1lA8/Uy446WojHZtMLbviVey77NvsXPtxCicvZdU9v++2bjMzaxgVkiM1WlOICGKkQgyNUc8eMTRGfWScGB6nvneI6ra9UK0/+cUFoe5y2n+rp4y6S6gnm+8upfPd2fLuIko8MnwrdWQyBdD75vPYd9m3Aaht3c/u532Gwc1vd6ZvZmYLQhLqLUNvmcLgwLTbRaVKfThNsGJ4PE22RirEyDj10UqahD12kBitQEyzk3KRpKuYJVcl1FVKmxS7s+euLOmaWF4uoaITsEbp2GRKXUUGN19GsryHR0/9BJVbdjL6nV/Q86ozWx2amZnZYSoVKSwtwtLep9wuIoixSppojVWI0SOP+uTpoTFizyFirPqEqxSfpJCgcjFLtopHpstFKGfz5SIqFyZNp/OUiu5sP8m8kilJFwOfAArAZyPiww2JqkHKz14DwAk7/4jHXvL37H31l+n5P59B/x//EuUNq1scnZmZ2exJadMf3bO/72xUa8RoNU2+xirEWDV9jGfPY5XD0/WhMWLvEDFehUpt5p0XE1Q6kmxRypKuUiHtVlMqpOsn5svps4pFKCWH11FM2r7VaM7JlKQC8HfARcA24CZJ10TEXY0KrlEKq/o57urXsusZn2LkS7cz8qXbWfrJS+h6ycloaTeFE/pb3p8qspF6p/tARa3+lHdLj/Fq+mGebt/VekvOMSq19AqX4vTHrg+Po55S2/8zmZnljYoF1F+Ao265NpOoB1SqxHgtTbae8Kg9cV2ldjgBqw+NpfOV2uwSsgnFAsoSrDRJK6TLnjCdbjMxTTHJlhXSWrZiIU3SstdRKKR9zxbgu2U+NVPPBe6NiPsBJH0FuBTIXTIFUDp3FWuqf8bQpzdx6CM3sP8d3z2yUpAc1wPdWRVnKYHIkpAgvSXBNM9RC1QQdBXT+89Ptd2k4yA98XmkSv3QOHFoHPWV0jgm/vAC6kF97wixf4xkVR/qLUGiNLESxFiN+oExYu8IyZqBtPq1mKBiQlTrMJbtf98oyQn9JANdUGjSB6uevh9U61CrE9U69T0jqJSk51XKPuBZ3GmfgCpxYAz1lymsGUj/gWt1qNYP7ytGKiTLe6BcSNfVA2oBiY6cy8T7XZ/0npcL6fsxWUzR4WCqRZP/hhP7rAf1/aMgpeeTKF1Xj/SzMF4jDoylsXYXn/TaifNNlnenn5fJ5zmxn4JQfzk9v0qNqNTTAurgOOotkSztSt/HROnxxqrpe3lwjGRlL+oupoVKoiPrhivEeI3C8X1QTNL3LIBJr6VcSOMupJ+tqNbTJoLRKvWD4yTLu1FfGRUTiucMsuIbr2vUp8bMckiJoCvtezVXEz/ko5ImXIzXsuk6Ua1lZVztidOVGlSz6bEqMTRpWbWWlo3HKku6KCaokNBz4dkUZmhSPeZDzOO1JwIPT5rfBjzv6I0kXQZcBrBu3bp5HG7+VEjo/93n0vc7z6F6124qt+wghivUHjlIfddQ+sUyVoVKferEZ9KzJuazL9QYq6ZfUNO9bpqETD1FNNCF+krU94wQw9kAcRPbJSJZ3k2ypIva9oNErZ5+mCa+fLuLJH0ltKQrPYeJZKZSSz843cX0sbyH+qOHiENTXKrbsDdY6Zd1MUkTzGJCsqybGKqkX+jVNHmYiFvdRdRTpHB8H7VdQ9R3HDr8Za9C9qVfSM+hvm80PadCtu+J933iH0vZP//Eex4c/ufl6F8lU+WSU/1yEelxsn0qEVrSBQH1vSPpNsmRdRQTNNCVrqvUjrw2yT4v3UXUUyIeH0l/2WVJb5oUZglQtZ7+jQrKqsnTX2XqLxND49QPjqf7rgd0ZdXrXen6+mPDxFjtyHvcVcje4xIqJdR2Dx9O4JCOfDb6SsRolfqBsfSzVa2nx+3K1veX04R+tArVOoV1HmrEzGYmKWvqa1yrSNSzH6HV2uHnqNSPJFvVejpkxaT1T3iu1Z+ypWSumt4BPSKuAK4A2LhxY5O+xY+NJErnHE/pnONbHYqZmZnNkpIEysm03VpaZT7XRW4H1k6aPylbZmZmZrZozCeZugk4Q9IpksrA64BrGhOWmZmZWXuYcz1ZRFQlvQP4/0iHRrgyIu5sWGRmZmZmbWBejY4R8V3guzNuaGZmZtahPJa8mS1aki6W9HNJ90p6b6vjMbP25GTKzBalSQMPvwI4G3i9pLNbG5WZtSMnU2a2WB0eeDgixoGJgYfNzI6JkykzW6ymGnj4xMkbSLpM0iZJm3bv3r2gwZlZ+1jQUa82b978mKSts9x8JfBYM+NZAO1+Do6/tTol/pNbHchcTR50WNLuYyi/IP9/vzzHl+fYwPHNV57jOzq2WZVfC5pMRcTgbLeVtCkiNjYznmZr93Nw/K3l+JvumAYePpbyC/J//nmOL8+xgeObrzzHN9fY3MxnZouVBx42s4bI181tzMwWiAceNrNGyXMydUWrA2iAdj8Hx99ajr/JmjzwcN7PP8/x5Tk2cHzzlef45hSbIqLRgZiZmZktGu4zZWZmZjYPTqbMzMzM5iGXyVQ73C9L0pWSdkm6Y9Ky4yRdK+me7Hl5tlyS/jY7ny2Snt26yA/HulbSdZLuknSnpHdmy9viHCR1S/qppNuy+D+QLT9F0o1ZnF/NrtJCUlc2f2+2fn0r458gqSDpFknfyebbLf4HJd0u6VZJm7JlbfEZapa8l19T/c1aHM+sy9IcxXe5pO3Ze3irpEtaFNsxleM5ii8v798xfY88pYjI1YP0qpr7gFOBMnAbcHar45oizhcDzwbumLTsvwDvzabfC3wkm74E+GdAwPOBG3MQ/2rg2dn0APAL0vuTtcU5ZHH0Z9Ml4MYsrq8Br8uWfxr4v7Pp3wE+nU2/Dvhqq/8GWSx/AHwJ+E42327xPwisPGpZW3yGmvR+5L78mupv1uJ4Zl2W5ii+y4E/ysF7d0zleI7iy8v7d0zfI0/1yGPNVFvcLysirgf2HrX4UuCqbPoq4DWTlv9DpH4CLJO0emEinVpE7IiIm7Ppg8DdpLfSaItzyOI4lM2WskcAFwBXZ8uPjn/ivK4GLpSkBQp3SpJOAn4V+Gw2L9oo/qfQFp+hJmmL8itPjrEsXXDTxJcLcyjH8xJfLszhe2RaeUymZrxfVo6tiogd2fROYFU2netzypqMnkWalbfNOWRNZLcCu4BrSWsE9kVENdtkcoyH48/W7wdWLGzET/I3wLuBeja/gvaKH9KC53uSNku6LFvWNp+hJmiHc5zqb5Y3032G8uQdWXP1la1shpwwy3K8ZY6KD3Ly/h3j98i08phMdYRI6wdzP+6EpH7g68C7IuLA5HV5P4eIqEXEBtLbgDwXOKvFIc2apFcCuyJic6tjmacXRcSzgVcAvyvpxZNX5v0ztEg95d8sb3L6GfoUcBqwAdgBfLSVweS9HJ8ivty8f436HsljMnVM98vKmUcnmi2y513Z8lyek6QS6Qf8ixHxjWxxW50DQETsA64DXkDadDQxGO3kGA/Hn61fCuxZ4FAneyHwakkPkjYFXQB8gvaJH4CI2J497wK+SVoYtd1nqIFyf47T/M3yZrrPUC5ExKPZl3Ad+AwtfA+PsRzPRXx5ev8mzPJ7ZFp5TKba+X5Z1wBvyabfAnxr0vI3Z1czPR/YP6kKtiWy/jafA+6OiI9NWtUW5yBpUNKybLoHuIi0Pf464DeyzY6Of+K8fgP4YfaLrSUi4n0RcVJErCf9jP8wIt5Am8QPIKlP0sDENPDLwB20yWeoSXJdfj3F3yxvpvsM5cJRff1+jRa9h3MoxxfUdPHl6P071u+R6TWqV3wjH6RX/fyCtO3yT1odzzQxfpm0erJC2qb6VtI+LD8A7gG+DxwXR64Y+LvsfG4HNuYg/heRVv1uAW7NHpe0yzkAzwRuyeK/A/izbPmpwE+Be4F/Arqy5d3Z/L3Z+lNb/TeYdC4v5cjVfG0Tfxbrbdnjzon/1Xb5DDXxfclt+TXd36zFMc26LM1RfP+YfYa3kCYuq1sU2zGV4zmKLy/v3zF9jzzVw7eTMTMzM5uHPDbzmZmZmbUNJ1NmZmZm8+BkyszMzGwenEyZmZmZzYOTKTMzM7N5cDJlZmZmNg9OpszMzMzm4X8DpyO8xvlpaeMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "# 定义网络结构\n",
    "model = Linear(2)\n",
    "# 定义优化器\n",
    "opt = RMSprop(init_lr=0.1, model=model, beta=0.9, epsilon=1e-7)\n",
    "train_and_plot(opt, 'opti-loss3.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**动手练习7.4**\n",
    "\n",
    "动手实现AdaDelta算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 7.3.3 梯度估计修正\n",
    "除了调整学习率之外，还可以进行梯度估计修正。在小批量梯度下降法中，由于每次迭代的样本具有一定的随机性，因此每次迭代的梯度估计和整个训练集上的最优梯度并不一致。如果每次选取样本数量比较小，损失会呈振荡的方式下降。\n",
    "\n",
    "一种有效地缓解梯度估计随机性的方式是通过使用最近一段时间内的平均梯度来代替当前时刻的随机梯度来作为参数更新的方向，从而提高优化速度。\n",
    "\n",
    "#### 7.3.3.1 动量法\n",
    "\n",
    "动量法（Momentum Method）是用之前积累动量来替代真正的梯度。每次迭代的梯度可以看作加速度。\n",
    "\n",
    "在第$t$次迭代时，计算负梯度的“加权移动平均”作为参数的更新方向，\n",
    "\n",
    "$$\n",
    "\\Delta \\theta_t = \\rho \\Delta \\theta_{t-1} - \\alpha \\mathbf g_t = - \\alpha \\sum_{\\tau=1}^t\\rho^{t - \\tau} \\mathbf g_{\\tau},\n",
    "$$\n",
    "其中$\\rho$为动量因子，通常设为0.9，$\\alpha$为学习率。\n",
    "\n",
    "这样，每个参数的实际更新差值取决于最近一段时间内梯度的加权平均值。当某个参数在最近一段时间内的梯度方向不一致时，其真实的参数更新幅度变小。相反，当某个参数在最近一段时间内的梯度方向都一致时，其真实的参数更新幅度变大，起到加速作用。一般而言，在迭代初期，梯度方向都比较一致，动量法会起到加速作用，可以更快地到达最优点。在迭代后期，梯度方向会不一致，在收敛值附近振荡，动量法会起到减速作用，增加稳定性。从某种角度来说，当前梯度叠加上部分的上次梯度，一定程度上可以近似看作二阶梯度。\n",
    "\n",
    "**构建优化器**  定义Momentum类，继承Optimizer类。定义step函数调用momentum进行参数更新。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Momentum(Optimizer):\n",
    "    def __init__(self, init_lr, model, rho):\n",
    "        \"\"\"\n",
    "        Momentum优化器初始化\n",
    "        输入：\n",
    "            - init_lr：初始学习率\n",
    "            - model：模型，model.params存储模型参数值\n",
    "            - rho：动量因子\n",
    "        \"\"\"\n",
    "        super(Momentum, self).__init__(init_lr=init_lr, model=model)\n",
    "        self.delta_x = {}\n",
    "        for key in self.model.params.keys():\n",
    "            self.delta_x[key] = 0\n",
    "        self.rho = rho\n",
    "\n",
    "    def momentum(self, x, gradient_x, delta_x, init_lr):\n",
    "        \"\"\"\n",
    "        momentum算法更新参数，delta_x为梯度的加权移动平均\n",
    "        \"\"\"\n",
    "        delta_x = self.rho * delta_x - init_lr * gradient_x\n",
    "        x += delta_x\n",
    "        return x, delta_x\n",
    "\n",
    "    def step(self):\n",
    "        \"\"\"参数更新\"\"\"\n",
    "        for key in self.model.params.keys():\n",
    "            self.model.params[key], self.delta_x[key] = self.momentum(self.model.params[key], \n",
    "                                                                      self.model.grads[key], \n",
    "                                                                      self.delta_x[key], \n",
    "                                                                      self.init_lr) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**2D可视化实验**  使用被优化函数展示Momentum算法的参数更新轨迹。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x1 initiate: [3.], x2 initiate: [4.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAF5CAYAAACFu8BrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XmcJOld3/nPE2feWZVZV989Mz2nRreEERhZgGVrgZVtzPICvDYv23h8YIONMYvAL2nxLja+sNkXxni8eA02IMugAQGy8EiMToSO0cGMNKO5p4/q6qrKyvuI89k/IjIqq7qqu6unq7Oq+/fW69FzRGTUkzUzFd98MjJSaa0RQgghhDiMjGlPQAghhBDiekmQEUIIIcShJUFGCCGEEIeWBBkhhBBCHFoSZIQQQghxaEmQEUIIIcShZU17AgBKqZeALhABodb6TdOdkRBCCCH2027nfqXU3wN+MB3/Pa31j13pOAciyKS+WWu9Pu1JCCGEEOKm2XLuV0p9M/DngNdqrT2l1MLVDiBvLQkhhBDioPjbwM9orT0ArfXq1R5wUIKMBv6nUupxpdRD056MEEIIIfbdTuf+e4BvUkp9Rin1MaXUm692kIPy1tKf1FpfSJeQHlVKPa21/vjkDumTfAigWCy+8b777pvGPIUQQoib7vHHH1/XWs/frJ+nlLoR31/0FWA00X9Ya/3wRP+ycz9JLqkBXw+8GXifUupOfYXvUzoQQUZrfSGtV5VSjwBfB3x82z4PAw8DvOlNb9Kf//znb/o8hRBCiGlQSr087Tlch9GVPryzy7n/PPD+NLh8VikVA3PA2m7HmfpbS0qpolKqPG4DfwZ4crqzEkIIIcR+ucK5/7eAb07H7wEc4IofBDoIKzKLwCNKKUjm82ta6w9Nd0qHTxzHRFFEFEWEYbhje6cyftxO9bUUrfVl/fHYZL1bAS5rT9Z7kf47tKXe3t6pGIaxpR63x2V7/0rFNM1d692KZVk7tg1j6q8zhBBiv+x47ldKOcB/Uko9CfjA91/pbSUAdT0njGm7Qe/dCSGEEAfKbudkpdTjN/MeazfoPHtT5nwQVmQE8Na3vhXLsrAsC9u2MU0zq8fj4zL5yn17vdsr/SutDEyuHOy0urB9dWI8vtuKxk4rHFcqcPnqyfXabZXnamX76tL28XE7iqLLVqF2W8WaXOHabSXsSitn47Gd6skSRRFBEFw2/vGPf/xKvyohhLgl3LJB5q1vfSuu62bFcZwt7XF/3B73bdvesT+uLcvKHmfb9pbt47ZpmtN++kJclyiK8H2fIAiyelx838fzPMIwxPf9rIy3TfY9z9syNtn3PC/rj9ue50nwEkJcl0MZZF7/+tfzyU9+ctrTEOKWY5om+XyefD4/7alcJo7jLPSMRqOs3t6+UhkOh5e1h8MhH/7wh6f99IQQ1+lQBhkhxO3HMIyphKwoihgOhwwGgyz4DAaDLXW/32c0GjEYDOj3+9m27eXRRx+9qXMX4nYgQUYIIa7ANE1KpRKlUmnffkYYhlkIGtfjsr3f7/fp9Xpb+p1Oh49+9KP7Nj8hDjIJMkIIMWWWZVGpVKhUKjf0uL7v0+v16PV6dLtd+v0+3W4360/WvV6PTqeTjT322GM3dC5C7BcJMkIIcYtyHIdarUatVnvFx4rjOAtCnU5nS91ut+l2u1vanU6HdrvNRz7ykRvwTITYnQQZIYQQV2UYBuVymXK5zNGjR6/7OFEU0el0sqAzbrdarWxsPD5uSxgSVyJBRgghxE1jmiazs7PMzs5e9zFGo1EWclqt1pa62Wxm/XGRt8lubRJkhBBCHCq5XI5cLsfi4uKeHxvHcfYW2MbGRhZ2ms1mVsb9jY0NPvaxj+3DMxA3kgQZIYQQtw3DMKhWq1SrVU6ePLmnx/q+n4WdjY0Nms0mjUZjS39jY4NHHnnkuucXx7F8z9oeSZARQgghroHjOCwuLu55JWgwGLCxsUGj0WBjYyMrjUYjKxsbGzz66KMSYq6DBBkhhBBiHxUKBQqFAsePH5/2VG5JEv2EEEIIcWhJkBFCCCHEoSVBRgghhBCHlgQZIYQQQhxaEmSEEEIIcWjJp5YOIa01cRwTRRFhGG4pURRtGR+3x+NxHG/pT47vVm9vx3GczWF70Vpn2ybryQJk4+PnM9merPdCKXVZPdkef6xxPD4uhmFsqcft7WX7uGmal7V3qieLZVlbxi3Lyupxe7I/WeRjmUIIcbnbPsjEcYzneVnxfR/f97N+EARZPwiCLfuMt+3UD4Ig64/rMAy3jI3LeHyyPVlPlvGYuD1ZloVt25eFnPHYZL29bds2juNk4+P2uB63J8d36ruui23buK67ZR/XdbdsGxcJYEKI/XQog8za2ho/+7M/i+d5jEYjhsNh1h6NRll7HEa2t8fBZDQa7UsoGP8x336C2OnE4gZQNF2sYgnbspJi2liWiWVa2JaNZZpYlolt2pjpuGWaabGy2jCNLX3TMJJX+YaBYZiY6XbTMDCNdJUgaxsoZWRjxnh8vAKhjHQfhWmYKENhqM1tSiV3zFRMrHBsW/lQjFc7tvbh8tWT67XbKo9mvFKkt/SzFaSJlaRYj1eRINbx5lisieIIrTVRFGfbojgmiqPN9uRKVhxlY1EcEaZ1FMXE6bYwDLPxMIoIo5A4igmjMOsnddKOwoggCgjDZCwIg2SfdDwIQ4IwJAwDwiDCVyHD4fCysLw9cI/LjWZZFrlcbku4GZfJ8XF7fPv5cTufz28ZH5fxeD6fz/rb27Zt3/DnI4Q4WNT1LOFPm1Iqm7Rt27v+AczlcuQCcGyHnOvi2g6u4+I6TtLOxpK+Yzs4tr3Znxh37fRVqW2nY2lAsZK2Y9vYloVlWq/oRCzENGmtCaMkCPlBkJQwXVUMArzAx0/Djxf4eL6PH6armGnfC9JQFAZZ3/O8ze1+0h55Hn7gM7K57EXIZDsIgut+PqZpUigUsnAzbk/W4/HJ9uTYlUqxWCSfz8uqk7hhCoXCjuNKqce11m+6WfOYPM++AjdlzodyRebVd9/PY//xfbi2g2ma056OELcMpRS2ZWNbNoVcftrTASCKoiz4DL0RI2/EyPcn2uPxpB6OkvGh5zHwhll7OBom29N2N+iyurrKcDhkMBhk9fWsSo1DTbFYpFAoUCqVsno8XiwWL+uXy+UtdalUyorruvvw2xTi1nMog4xlmgfmj6wQYn+ZpknBzN+0/+ajKGIwGjIYjRh6w6w9GA7oj4ZZ3R8OGAyHW8YGwwG94SCth6ytrTEYDOj3+1m5VrZtbwk241Iuly+rx+1KpXJZPQ5JsmokblWHMsgIIcR+MU2TcrFEuVi64ceO45jBaEhvOKA/6Kf1gN6wT384oDvo0+v36Q379AZ9eoMBvUE/GR/06Pb7rKys0Ov16PV6dLvda7rOTymVBZ5KpZIFnMl6e7tSqVCtVre0c7mcvHUuDhwJMkIIcZMYhkGpUKRUKEJ9/hUfT2uN5/t0+j16g15a93esu/1eWnfp9Hq0223OnTtHp9Oh0+lc02qRbduXhZtxv1qtMjMzk43t1C6Xy7IyJG44CTJCCHFIKaXIuS4512WhVn9FxwrDkO6gT6fXpdPv0emnda9Lu9dN+ml7XDq+x7PPPku73abT6dDr9a4633G4GQeccalWq8zOzm4Zq9VqzMzMZOOWJacscTn5t0IIIQSWZTFbqTJbqV73McIwpNPv0e51aHU7E6GnQ6vTptXr0u52aHXbyfZwMwi1Wi0Gg8EVj1+pVJidnd2x1Gq1LfW41Ot1HMe57uckDj4JMkIIIW4Iy7KoVWeoVWeu6/F+4NPsjINOm41OOwlA3Q4bnRatbodmp8VGp01zMGB5eZlGo0Gr1britUKlUolarZaFnHq9nvXHpV6vZ+P1ep1yuSzXAx0SEmSEEEIcCI7tsFifY7E+t6fHaa3pDno02202Oi2anTbNTotGu8VGu8VGp5nWbTb6fc6dO8fGxgbNZnPXr0OxbXtLwNmtzM3NZXWhUJDwMwUSZIQQQhxqSikqxTKVYplTR49f8+OiKKLV69BoJUGn0U4DT7tJo91kfTwe9nn66adpNBo0Gg3iON7xeLlcLgs243BzpTI7OysXP98AEmSEEELclkzTpF6dpV6dvebHxHFMu9el0WrSaG+w1tyg0W7SaDVZb22wntaNbpcXX3yR9fV1ut3u7j+/Xmd+fj4r733veyXc7JEEGSGEEOIaGYaRXRR9htPX9JiR57He3mC9uTEReDZY3Wik9Trrwx5PP/20hJjrIEFGCCGE2Ec51+X4whGOLxyZ9lRuSRL9hBBCCHFoSZARQgghxKElQUYIIYQQh5YEGSGEEEIcWgcmyCilTKXUF5VSvzvtuQghxEHy3g99gHv//FspvuVu7v3zb+W9H/rAtKckxA2x/dyvlPpWpdQXlFJfUkp9Uil15mrHODBBBvhh4KlpT0IIIQ6S937oA/zgz/wE51aW0VpzbmWZH/yZn5AwI24V28/9/x74S1rr1wG/Bvzjqx3gQHz8Wil1HPh24KeBH5nydA4VrTVRFBFGEWEUXlZHYVrHcTY+3j+Oo6wfxTHRRD/WMVEUo9M66UfEWhPFEXEcJ0Vr4vSxOtZokr7WmljrpI7jifFkzuP++DlodLZtPLZX41uDK6VQChQqGzMMI+srtdk3DAOlFIZKtmX7GQrTMDEMAyMdNwwjGVMK0zQxlIFpGqi0NpSBZVqYZrKfaRiYpollmhhGUo/7lmlhGun+VtIfj49r27Ky+Ynb13t+8V8xHI22jA1HI97zi/+K73nHO6c0KyFeuV3O/RqopO0qsHy14xyIIAP8W+DHgPJuOyilHgIeAjixdHTfJxRFEV7g4/l+Unte1h/5Hn7g4wdBNuZP1oFPEAT4YZCOBQRhgB+kJUy3p/sEYUiYbg/CkCAM8UOfMG0nZbxfSBglY2EUEYTBvv8ubial1JZAcq0mA9D1hKCDzLZsLNPEtqwk6FhJyLEtO62TMcdysr5j21jpdseycWwb27ZxLAfHTvvpuOs4yT7j2naSsbR27aTYtk3OcTfHXDepHYec48qNvPbJ+UsX9zQuxAEyp5T6/ET/Ya31wxP9nc79PwB8UCk1BDrA11/th0w9yCilvgNY1Vo/rpR62277pU/+YYD777xbf/bJLzL0Rgw9j9FoxMAbMvQ8PM9Lx0eM0vbI9/B8LxvzfH/LeFL7aTsJJDcyIJimiWunJxArPaGMTySWnZyE0r7V9sibFrZZwLJNbNfCNk0sw8Q2klfrtpH0LdPEUia2aWKOx7JiZGOmMrJ+0k5WGrJxZUz0k5UEpRSmSrcpIxszxuNGMq6UwkBhqHTlYryyodTECsi2PqSPTVZN0n8PbtjveywLN+lqT6xjNBMrQmno2d6PdJztH5OuKuk4WbXS8eZ+cbpSlZbxWBhHyX7p/uH4cRPbojjKxsM4yvphHGUliiOCKCLU6YpbHBGkq2ZBHCb9tB1EIcEoIow8olmDdq+bBd8kLKdBOQ3P4xAeRdEN+33blp2GGgc3DTxJ8HHJOS55N5eMueN2UuccNxvLuzlc16WQtnO5HHnXJe/mybs5CrkcOTdHIZenkMvh2M4tv2J1fPEI51Yuf1F6fFFurib2z+tf/3o++clPvqJjFIvFda31m3badoVz/z8Avk1r/Rml1D8CfpYk3Oxq6kEG+EbgnUqpbwNyQEUp9V+11v/7bg94+sXneNsP/G9XPfD4D2lu/Ecx/QObd3PYHZ+6ZePaBXJ5B9eycU2bnLXZdiwL13RwLQvHtHEtG8e00m1b265pYY/7ZrLNsSxsI3kLQdx82coOChSYB+qSsIMhimOCOMQPQ/woxI8CvCggiEK8KAlBXhQk2ybaXhgk+4YhXuTjhyFeFOCFAaPQ39oOAryhx6CiabSb2QuG5IVF8qLDD67vhYNhGFnAybtJuCnkCxP9zbFiLk8+l6eYz1PMFcjncpTyRQr5ZKyQK2TbioUCpXwR15l+UPqpv/Wj/J1/+i5GvpeN5XM5fupv/egUZyXEK7bTuf/3gPu01p9J9/lvwIeudqCpBxmt9buAdwGkqexHrxRiAE7NLPLP/sxfJW855G2XnO2QsxzyVlInfRtDyYlLiCsxDQPTSP67maZYx4zCgFHgMwp9hmFSjwKfYeBl/WG6fRB4DAMvGxsEI4Zpe+h7DPp9Nkoe5y8tMxiNGI6G9EdDBqPhnt56NAyDUr5AIV+glC9QzBcoFYppXaCYLybjhQLlQpFSvkipUKRUSPYrF0oU8wXKxWLaL+I67p5+N9/zjnfyx89+lX/7q/8vCsXxpSP81N/6Ubk+RhxqO537gT8PrCil7tFaPwO8nWv4ENDUg8z1mMkVefuZN0x7GkKIG8RQBgXbpWDv7SS/V1prRmHAIBgxCDwGvpe1+4HHwE/HA4++P2IQjOj7I/ppPQg8RqZJo93k7MXz9IYD+sMB3X6fMAqvaQ62ZVMuFCkXS1SKpSTgpO3JuloqJ/1CKbsw/vd+/le49/RdVEtltNZTXy0S4kbSWodKqb8B/KZSKgaawF+72uMOVJDRWn8U+OiUpyGEuEUppcjbDnnboX4Dj6u1xo9Cev6Qvj+i54/o+cOtfW+YjiXbut4gqdtD1nyPFy+cpdPv0el1GXqjHX/Ot/3dv5y1LdPaDDulEjOlCpVSmWqpTDVtz5TKaV2hWt4cq5YrVEtlbMu+gb8FIa7P5Llfa/0I8MheHn+ggowQQhxGSqnk2jrLpl6oXP0BVxHGEV1vyPue+Bj/12O/hhdtXkNkGybffu/XcWp2kc4oCUPtUZ9+HPPy8nnavS7tXodOv3fVt9GK+QLVUpnZSpVqqbKlPVOpMFuuMlOuUi2Xs/ZMpUKtMkPezcmKkDgQJMgIIcQBYxkms/kSv/jZ39sSYgCCOOLzF57lP/6Ff3DFY8Q6pueN6HgD2l6f9qhPZ7TZ7noDWqPN8Z5lsLx2ia++8Cytbpt2r3vF4zu2zUy5ymy5ymxlssxk7VrWnqGW1tVSGdM0X/HvSIgxCTJCCHFAXeg09jQ+yVAGlVyBSq7Aceb2/LOjON4Sdlqj3mXt1jCtOz2WvUt85flnaHXbdPq9XY+rlMrCT606w2xlhnp1hlp1llp1hlplhvq4XZ1hbmaWerVGPpfb83MQtwcJMkIIcUAdq9Q531nfcXy/mYbBTL7ETL6058cGUUh7NKA16tEc9miOerSGSbs16rEx7NIcdmkOeqxFDb720vNstJt0B/1dj1nI5anPzFKvzjI3M8vcTI36TI16dZb6zCzzMzXmZmvU0221yoys/NwmJMgIIcQB9ZNv+15+5IP/gWHoZ2N5y+En3/a9U5zV1dmmxVyxwlxxb9cL+VGYBJ9hl41hl41BN233aAw6ydiwS6vX5YULZ1lvbuy6+qOUolaZYW6mxnytxtxMjbmZOnOzNeZnaszP1tPxOvOzdepVCT6HlQQZIYQ4oL7rwW+iOezyE4/+ZwCOV+b4ybd9L9/14DdNd2L7xDEtFkszLJZmrvkxfhSyMezSGHRo9Ds0xu20rA86tIh5+sXnWG99lka7teNF0Eop6tWZJODM1lmozWXt+VqdhVqdhXR8oTZHMV+4kU9dvAISZIQQ4gB7/dEzAPzqd/84f0bun3UZx7RYKs2yVJq9pv2jOM6Cz3q/zXoadtb7bRqDDmv9Ns0o4svPfJW1ZoNWt7PjcQq5fBpqNsPNuL9Um8/6i/U5SoWifMJrH0mQEUKIA+xidwOAI+XalGdyazANg/lilfliFeZPXHV/LwyygLM2aLPWb7PeT+q1fou1XpuXhuf47BNfZK21seNqzzj0LNbnWExDzmJ9jqW5BZbq81m9UKuT348nfYuTICOEEAdYFmRKEmSmwbVsjlbqHL2GC6yjOKYx6LDab7HWb2f1Wr/Faq9Fw415/vxLfOpLn6PRbl72eMu0GHkjuVZnjyTICCHEAbbS3cA2TOqF8rSnIq7CNAwWSjMsXMM1Pn4UstZvcanX4lKvyWqvRccbSIi5DhJkhBDiAFvpNVkq1+Qai1uMY1ocq8xxrLL3e/yIreTroYUQ4gC72N1gSa6PEWJXEmSEEOIAu9jd4Mg1fiJHiNuRBBkhhDigfuPJj/PCxkU+8PQf8fqf/zv8xpOfmPaUhDhwJMgIIcQB9BtPfoIf+eDDjD/Me76zzo988D9ImBFiGwkyQghxAP30R399y1cTAAxDn5/+6K9PaUZCHEwSZIQQ4gB6Jd98LcTtRIKMEEIcQLt9w/XN+OZrIQ4TuY/MLURrTaw1QRwSxjFhlNbjfhwRxVHS1lHaj7M6iiMiHRNOtKM4JtIx8bjWOq2TsTj9meMxrUGzOR6njxnPDXTW1+mcNTp9XPK/5LlMPC8uv+X3mGLz3hrj22yo8f9U2lbJXkopDKWApB73DWWktUJhoBQYysAcjxtGtt94zFQGhpH0zaw2sSbapmFgTdSWSrcbZtI3DCzDSmozqW3DyuYmbm8/+bbv5e/97i8QxlE2dhi++VqIm02CzB7FOsYLQ7zQZxQF+GGAFwaMogAv9JNtkY8fhoxCHz8K8dL9xu0gGwvxozDrB1Haj0O8MCSMN7cHcZRtD9N2GEfb+vEVT/oH0TiIbA8dyZi6bL9Jk891/P0m28PR9v0OA4VKQo1pJSUNPpP9ydpJxx3TwjY2+65pZ+OOtdl3TTsds3FNGzetk30cXMvCtRxy6TbHstN2ss1QspB7M3zXg9/Er335D/jU2a+idbIScyt/87UQ1+uWCDJ+FDIMPAZpGQYew8BP6tBjkLZHoc8oHI/7DAOfUZCOpdvG/VEYMAp9vHSbl/b9KLwhc7YME8e0cC17x5OPbZo4po0TQdFwcSwrOZlNFMswccYnuclaTa4AjF/5b46byshWDrK+MjDScUuZGEay6mCm4+Z4FWLbysR4dUIphcH21Y1tdbrPZGi5WSbDTczmCpFmaz1eXdLpPuNVqC0rUWk7SlecIj1etdLpSleyX5iOh5MrXWk/HNfxuJ+skAUT/SDeLGF8ed+PQ4IoIlQxg6BPEEWXhWI/CtI63PLK/pXIwo9lk7ecJPRYDjk7CT85O+nn0/G8nWzLWw55203HkrpgO+Qtl4LjkrfdpG0n2wu2i23eEn+irlvednlg4RSP/fV/Me2pCHFgHcq/Ek+tneXB/+dvZuHlev5AW4a5+cc2/cOb/TGOFTN2kVx+Fte0yJsOrmmRy+r0lWwaPHJp28nGLBxjoj3RdwwLxzTlVe1NptJAhYLb9ZtMYh3jR0kA8qIQPx6vEm72s3aUtEdRgBeNx9OVxyhkFPl4UcgwrUfpCuWIgNaon71oGAWbLxKu97/Tgu1mwaZguxScHMXJvp2j6EyUXfolJ0fJyVNy8ziHJCAtdzc4Knf1FeKKDsd/zdsUTIc/feQBCmkQGddb2ubmWM60s2050yZnOdjG7Xo6E7crQxnkLIMc9lR+fhBHWeAZjFdBoyBZHQ19hlFSj7dt1l7W7qftbjzkUq+ZrML6Hv1gxCDwrnkutmFScvNJsEkDTnHcdvOUnTxlN0/ZLVBycpTdAuU0BI3b4+37GYoudhq88eiZfTu+ELeCQxlkTpbn+Nff+JemPQ0hxB7Yhont5CmT35fjxzpmGAYMQo9+4NEPPXrBiH7g0RuPBaNsvBeMtyftjtdnubNOzx/R84d0veE1XV+Vs2zKToGSm6fiFrJSzm22q7kC5XHbLVLJFai4RarpPju9hTYKfRrDLkfK8iklIa7kUAYZIYTYzlAGRdulaLvM34CspLVmEHr0Ao9uMKIbDOkHHh1/mPW7/mQ9SrZFAevNi3S8AR1vQNcbXvVnFWw3DTxFKrkiVbeAla4af2H5OX7hM79DxS0yk0tKNV/K2iUnL59yE7c1CTJCCLEDpVRyfY2dY5HqdR8nimN6wYhOMKTrD2n7Q7pBUnfS0vYHaT2k4w9Y67e51GsC8Ohzj/M/n3t81+ObykhCTb7ETK6U1sWknSsymy8zm0/GZ3OlpJ0rMZMvZmFJiMNMgowQQuwj0zCougWqbmFPj3vkhc/xtz/+//HRP/ePOVqcpeUPaHuDpN7WbnpJv+0PaAw6PL+xTGvYoz0aXPHtsYpboJYGnVqhQi1fopYvT7Qr1AvlLWO3+yfJxMEj/0YKIcQBtDxoAXCsVKNk5yg7eU6U9na9TKxj2v6Qlten6Q1oeX1a/oANr0fLG9D0+kkZ9Vnvt3l2/TyNYZe+P9r1mBW3QL2QBJx6oUItX2auUEnHkjLZLzq5V/R7EOJqJMgIIcQBtNJvUbJzlOzrDwKGMph1i8y6Re7Yw+O8KKDp9dkY9djw+mx4fRqjLk2vT2PUY2PUozHqcaHT4ImVl1gftHe9x1bBdi8LN/PFKnOFCnPFatpO6nqhcmg+Gi8ODvk3RgghDqDlQYsjhZmp/GzXtFkqzLB0jT9fa00/9GiMejRGXdbToLM+6tIYdbP2ar/FU2tnWe938KJgx2PN5IpZsJkvVpkrVlkoVpkvzmRj43bBdm/k0xaHlAQZIYQ4gC72mxwtTifI7JVSKls9OlWeu+r+Wmt6wYj1UTcpwyT8rI+6rA07rI26rEdDnlo7x9pLT9Aa9Xc8TsnJM1+sslCcYaE0w0IacBZKMywWZ1gszbJYmmGuWJULm29hEmSEEOIAWhm0uHvmvmlPY18opZKbCjp57qgsXHV/PwrTkNNlbdRhddhhfdhhbdhN2vGIp9fO8fGXnqC9Q+hRKOaKFRZLsyyVZrN6qTzLUqnGUjkZm5fAcyhJkBFCiAPmvz/3RywPWvy35/6IT118hne94Z38xbu+btrTmhrHtDhanOVocfaq+47CIAk7gw5row6XBm0uDcd1m9V+iz9eeZH1QZtYb/1El0IxX6xypFxLQ84sR8o1FtOwc6Rc40ipxmy+JPfuOUAkyAghxAHym89/ln/06V/P+uf7G/zoH/4awG0dZq5VzrI5Uapf9RNeYRyxPuqyMmhnIWdl0GZl0GJl0OZ8e53Pn3+GxrC7489YKtWSYJOVOkfKNY5Wahwt11kozcjqzk0iQUYIIQ6Qf/aFDzDadiHsMPL5Z1/4gASZG8gyzGu6oNlahWnSAAAgAElEQVSLAlaHHVYGLS4O2qz0W1wctJLAE/b5wsXnWHlmg1G49Z+ZoRQLxRmOVpKAc6wyx9FyEnKOVOocq9RZKs3KfXluAPkNCiHEAXKhv7GncbG/XPPqKzxaa5pen5VBi+VBi4tp2FnuN1kZtHiusczHXnyCnr/16yoUioXSDMcqdY6Wk3Dz8z/1F7AsOTXvhfy2hBDiADlWrHF+h9ByrFibwmzEtVBKUcuVqOVKPFA7vut+XX/IchpwLvabXOg3WR40We43+dr6eT534Rl+UULMnk39N6aUygEfB1yS+fyG1vo9052VEEJMx7ve8E5++JO/QqjjbCxvOrzrDe+c4qzEjVB28tzr5Ll35si0p3JgKKVM4PPABa31dyil7gDeC9SBx4G/rLX2r3QMY/+neVUe8C1a69cCrwPeoZT6+inPSQghpuIv3vV1vLp+AksZKOB4sca/+obvk+tjxK3qh4GnJvr/HPg3WuszQBP461c7wNRXZLTWGuilXTstu3/LmRBC3OIMZfANS/fwvj/7Q9OeihD7Ril1HPh24KeBH1HJZ9q/Bfi+dJdfBv5P4N9f6TgHYUUGpZSplPoSsAo8qrX+zLTnJIQQ07Lcb17TPVOEOOT+LfBjwPh91DrQ0lqPv7jrPHDsageZ+ooMgNY6Al6nlJoBHlFKPai1fnJyH6XUQ8BDkCy1CiHErSiMIy4N2xJkxHQNA9QTF1/pUeaUUp+f6D+stX4YQCn1HcCq1vpxpdTbXskPORBBZkxr3VJKPQa8A3hy27aHgYcBXjt3St56EkLcklbSO84elu9ZEuIK1rXWb9pl2zcC71RKfRuQAyrAzwEzSikrXZU5Dly42g+Z+ltLSqn5dCUGpVQeeDvw9HRnJYQQ07HcbwJwtCArMuLWpbV+l9b6uNb6NPA9wB9orf8S8BjwXelu3w/89tWONfUgAxwBHlNK/THwOZJrZH53ynMSQoipyIKMvLUkbk//B8mFv8+RXDPzS1d7wNTfWtJa/zHw+mnPQwghDoILgyTIHJMgI24TWuuPAh9N2y8Ae7rXwEFYkRFCCJFa7jcp2znKTn7aUxHiUJAgI4QQB4h89FqIvZn6W0vi6qI4xotD/CgkiEO8tPbjiCAK8be0I8I4wo9DwjgimCjhZK2TdhTHSa1jwjgiHNdxTKzjrD9uRzomitNax8RaZ2MxmlhrtNbZeIxGp/tpINaaeKKvdZzUaHT6WbTJ9ri/G4XabKukr9TmNgUoZWCopG1saSuUMjCVwiAdVwpDKQwUpmFgpvubKmmPxyxlYBpm1jaUgWUYWIaJpZLaVAa2YWIaSW0rE8swsY3NerNYW/umhWOYOIaVtW3DwjUtbMPCMS1cw8I05LXIreaCBBkh9kSCzB7EOmYUhQxDf7NE/pb+KAoYRkHWHoUBo8jHi0JGoc8oCvGigFEUZLUfJeHEj4Nsux+FWXiJJr5z5UazJ06845OvZRiYykxrY2IsPWFPnNDt2TLuuG8k2w1DYSgjDQVbxybDwng7JF+6ppTKgolSydjYZGAZmww4WifhZzym00A1/uc2DlfjOtYxcTwOVePtyVgS1KJse9DsEsQBUZyEuTgNceFEoAuzMJj0k5CYBMMgjvbtn5+pjCzUuKaNY1rkTAvHsHFNK+3buKY9UVvkLAfXtMiZDjnLJp/VyX55y9ksprOlnzOt7J+buPGW+01eUz8x7WkIcWjckkEmimP6oUcvGNEPPPqhl9WDbf1+4DEI022hzyD0GAQ+/dBjGPoMspDiMYyC65qPbZjkspOMQz49obimRaE+Q9F2cC0b13ZwLSepbRvHstN+0h4X19radywL20xqx7KxTCsdS8Zty0zbFpY5HjcxDXNLWBD7Q2tNFEcEUbJqFkYhwbiEEUEUJKtpYbLNDwP8MCSIkjrpJ8Ub14FPEIV4QYAX+ulYMu6FfloHjHyPzkYbL+oyipIgPYp8/ChkFAXXHbLGAaeQhpuC5VCwXAp2UhctNxsr2kl/XBe29Ut2Lmvf7itMXhSwPurKiowQe3Aog8z5XoO/+dFfohcmQaUXjOgF4+Ay2lPgcAyLguVQtN30j3JSLx49QsHNU3BcCm6evONSzOoceSdH3nbJuy55J0fByZFznKS2XfJOUnK2g2Ueyl+zuEGUUlhpiMzjTns6W4RRyND3GQUeQ99jFHgM/BEj32fgjxj6I4ZeOhZ49L0RQ9+j7w0Z+h4Db8ggrburDdaGXYbhevoCIXkh4Mfh1SeSyps2RTtHyc5Rst20Tko5DTxlO0/ZzlFyNsfHF8eW7RwVJ0/Rcg9lSL/YbwFyDxkh9uJQnmE7/pAnNs5lf+xOnDxJOVekmMtTyhUo5wqUcgWKbtIvuXlK+QIFJ0cpl6fo5inm8hSdPLZ1KH8FQtwQlmlRzluU84V9+xlBGNL3h/RHQ/rekF5WD+iPhvTGbW9IdzSgN1HaK2usDtu80Fml64/ohSOGoX/Vn2kolYQdO0/FyVGx81ScpJSdfNavOoW0zlNxCltqZwovQC705aPXQuzVoTyLP3jybj72nv847WkIIa6BbVnMWGVmCuUbcrwwCul5Q7rDAd1hn+6oT2fQp5PWWX/YpzPs0R0OaA97rKyu87XWRbrBiI4/vOq1Z3nLYcYpUE2DTdVN2jNOgRm3SNXNZ9tn3SIzbpEZN9luGeZ1PbflgdwMT4i9OpRBRghx+7JMi5nCKwtGWmt6oyGdYY/2YKIMe7T63bTfpTXo0Rp02Vhe4UKvyVf8C7T9Ab1gdMXjV+w8M24ScGpuiVquyKxbZHaiXXdL1HKlbLtr2tldfY9IkBHimkmQEULcdpRSlPMFyvkCx2oLe358GIW0Bz2a/S6tfpdmv0NrkNTNfpeNXpuNXoeNXof1lRVe6K6yMerRvUIAcg0ru57o/l//R7y2fpI3zJ+mnitTz5WouyXm0vZcvnxorwMS4kaTICOEEHtkmRb18gz18t6+odoPA5r9Do1um41+h0a3RaPX5mNffZzf/cInshsKeFHA51af54trLxLs8haYa1jM5dOQkyszl5b5/GS7krTzZezrfLtLiINOgowQQtwkjmWzWK2zWK1vGf/Z3/uvhNs+Cq+Bpdo8n/m/f4X1bov1botGWq91WulYk/Vui5ULyzzTWmF92MHb5VNis25xS9BZyFdYyFeYS+uFfIX5XEVCjzh0JMgIIcSUnW+s7jpedJNPWp6aO3LV44yv/VnrNlnrbLDabqbtJqvtDVY7TS6dO88TjXOsjbq7XutTc0ssFtJwk6+wmK+ykNaLhSpLhSoL+SpF+2DdTkDcniTICCHElB2vL3CucWnH8b2YvPbnzoVjV91/4I1Y7Wyw1mlyqb3BameDS61GEnjaDS6eO89z7VXWhp0d7wdUsfNZsFksVDlSmGGpMLOlns+Xr/tTXEJcCwkyQggxZe/+zof4u//5n+MFm/fIyTsu7/7Oh/b15xbcHKfnj3J6/ugV99Na0+x3udRucKm9wcXmGhdb66y0Giy31lg+e45PrzzHpUGLcNs1PYZSLOarLBVmOFrcDDlHi7McSeulQhXXtPfzqYpbmAQZIYSYsu9+y9v58tln+Pnf/28o4Hh9kXd/50N891vePu2pAclKT61UoVaqcP+xO3bdL45j1rstlltrXGyus9xcz0LPheYqz164wMeXn97x01tzuXIadGY5WkzKsbQ+WpjlSHFGrt0RO5IgI4QQB8Dp9BqYr/2bRy67GPiwMAyDhWqNhWqN1526d9f9usMBF1trXNhYY7m5xoWNVS40V1lurnH23Fn+6NKztP3hlscoFIuFCseKtSzkHM/aNY6XatTconwk/TYkQUYIIQ6As41LOJbNfPnWvxlech3PKe45cmrXfXqjARc2Vjm/sbqlPvv8C3x14zyPnnuC0bbv1ctbDscngs2JUp0TpTon03ohX5GgcwuSICOEEAfAucYKx2sLGLf5N4CPlXIF7j16mnuPnt5xu9aajV6bcxuXON9Y5VxjhfONS5xtXOLsyy/xxNlzNEa9LY/JmTbHi2nAKScB51R5jpOlOU6W68w4BQk6h5AEGSGEOADOb6xyor407WkcGkqp7KaEu72NNfBGnF1f4WzjIi+vXeTlxgovr13kpZde4EuNl2l6/S37l+0cp8pzWbjZbNc5XqrJBckHlAQZIYQ4AM6tr/CnX/0npj2NW0rBzXHfsdPcd+z0jtvbgx4vr6chZ/1i1n7m5Rf58Lknt9xcUKE4WpzldBpuknqe0+V5TpfnqLr79w3y4sokyAghxJR5gc9Ku8HJOVmRuZmqhRKvOXk3rzl592Xb4jjmUnuDl9aWeWltmRfT+vnnnuV/nnuC9VF3y/6zbjELNacrScC5ozzPHZV55nJlectqH0mQEUKIKTu/kdzZ93htccozEWOGYXBkdo4js3O85Z7XXLa9Nxrw0tpFXly9wItrF3hxdZnnvvY1Pr/2Ir/90uPEWmf7luwcd1YWuKuykNTVxaxfdvI382ndkiTICCHElJ3fSO7qe6IuQeawKOUKPHjiLh48cddl2/ww4Oz6Ci+snueFSxd4fvU8zz71NI+vvchvvfg4ms2Qs5CvcFdlkbuqC9xVWeQn/L+O4zg386kcehJkhBBiys6tj4OMvLV0K3AsmzNLJzizdOKybV7gJ6s3l87x7MpZnls5xzPPfI3/cfbLjMKA99hyQfFeSZARQogpO9dYQSnFsdr8tKci9plrO7tegNwdDuRamusgNywQQogpO9e4xFK1jmPJq/HbWTkvn3y6HhJkhBBiys42VuT6GCGukwQZIYSYMrkZnhDXT4KMEEJMURzHXNhYlRUZIa6TBBkhhJiiS+0N/DDghNwMT4jrIkFGCCGm6FxjBYATcjM8Ia6LBBkhhJiisw25GZ4Qr4QEGSGEmKLNu/rKW0tCXA8JMkIIMSXv+/Sj/MsP/DIAb3n39/O+Tz865RkJcfjInX2FEGIK3vfpR/mhX/4XDH0PSG6K90O//C8A+O63vH2aUxPiUJEVGSGEmIJ/8v6HsxAzNvQ9/sn7H57SjIQ4nCTICCHEFJxvrO5pXAixs6kHGaXUCaXUY0qpryqlvqKU+uFpz0kIIfbb8frCnsaFuJUopXJKqc8qpb6cnvt/Kh3/VaXU15RSTyql/pNS6qpfQDb1IAOEwD/UWj8AfD3wg0qpB6Y8JyGE2Ffv/s6HcG1ny1jecXn3dz40pRkJcVN5wLdorV8LvA54h1Lq64FfBe4DXg3kgR+42oGmfrGv1voicDFtd5VSTwHHgK9OdWLiMnEcE0QhQRQSxhFRFGXtIIyI4nGJieKYMI6IJ/qxHteaKI7QWhPrGK1Ja02sNVprNBoArdN6h/mMv+x+/LX3CoVhKBQKpRSGMlCKtFaYhomhFKZhYCgD00iKYZhYhpn1TcPENExsKxm3TQvT3GzbpoVhHITXAOIw++63vJ3PvfAVHv7I+1HA8foi7/7Oh+RCX3Fb0Mkf917atdOitdYfHO+jlPoscPxqx5p6kJmklDoNvB74zA7bHgIegtvvxlFRHDHwPIb+iIE/YuT7DAOPke8l/cBj6Pt4gc8o8PECj2Ew7nt4gY8XBkkdBHihjx8GeEGAH6YlCvEDHz8KCcIAPwzxo4AwDAnSwBLreNq/igPDUEYaakxsy8Y2LRzLwrZsHNPCsZ2ktmwcy8a109pycG0b13ZwraTO2S452yFnO7hp27Ud8o5D3smRt11yjkvedsm7LgUnR95xyTsupmFO+1chXoGjs/MAnPt3H6KcL0x5NkLccHNKqc9P9B/WWmdXsyulTOBx4Azw77TWn5nYZgN/Gbjq5SYHJsgopUrAbwJ/X2vd2b49ffIPA7z+9H07vUCfujiO6XlDeqMBvdGAblr3RgP6oyE9b0h/NKTvDel7o6QeDen7QwbeKG2PGHhDhmlIGXoeXuhf95xcKzlBOrad1JaNa9k46QnVfK5HGQsHE5siNiYuJhYGNib2RHt7bWJgTRQzK2pL25gYU6h0TKW97TVpD9jy/2SjwOaKzbYRjSa+rJ3UMZoo7UfERGjitE76STskJiYm3FKirB0QEeqYIIwIwxjfi7LtHhFBWjx8AoaMzpRoD3r44USg3BYsX8k/38lwU3DzlNw8BTdHwc1RdPIUc3mKbp5SWhfcHCU3GS/lCpRyBcppXcoVKLl5WXG6SV5ev0itVJUQIw6cuDWg/4EvvNLDrGut37TbRq11BLxOKTUDPKKUelBr/WS6+ReAj2utP3G1H3IggkyavH4T+FWt9funMQcv8OkM+7QHPdqDLq1Bj86wn5Ss3aM7HNAZ9ekOB3SHfboT7Z43vOafl3fc7KRSdPMUnBzui0MWcShQJY9NAYc89paSw0rrpL1T7WLhpuHECFVyFdK1T03caM9deXOMxifCI8QjZETAaJd6mLaHaXsYJmXQ9xkSMCBgQIfWHQHLzTX6XhqS03B8rUpunnK+SDlfoJwrZu1Krkgln/Qr+SKVQimp80VmCiWqhTLVdGz79R/ici+tLXN6/si0pyHEVGmtW0qpx4B3AE8qpd4DzAN/81oeP/Ugo5ILHH4JeEpr/bOv5FijwKPV79Lsd2n2OzT7HVr9Xlp3aQ2S0h70sn4SXHqMgiu/KlZKUcmlf8zTP+TlFwOOUqZEnTIuRVxKOFldwqWIQzFtF9J2ARvTN8AHuq/kGYtbgYFKQ+gN/M/xxcuHImIGBPTxGeDTw6O3rd0f155H1/PotXy6eAzPjFhtN7Jw3x0NsuuXdpOzHaqFEtVCiZlCmZlimZk06Iz7s8UKs8Wt7dli5bYJQS+vXeS1p+6Z9jSEuOmUUvNAkIaYPPB24J8rpX4A+LPAt2p9bdczTD3IAN9I8j7YE0qpL6VjPzF5wc925zcu8f2/8B42+m2avQ4bvTbNfpeBP9r1hyilNv+AFsqUXw64hxJl6lTJUyVHZUtxKU/URe1gDJWsbIhDy8SgjEsZd+8P3raqFKPT8OPRwaPDiC4j2ozopv12MKLdHtJpe7QZsHoq5JmLZ7MXEFcKQgUnR61UScJNqUKtWKVerlIvJWWuPEO9PEO9nLZL1UMXfqI44lzjEu9845+a9lSEmIYjwC+n18kYwPu01r+rlAqBl4FPpx/keL/W+p9c6UBTDzJa60/CxMUP16DZ7/KV889TK1VYOmdwP8eYJc/MtlIll9Z5ytrF6Cvo79MTEeI2YqCy0H/0Wh/08mYzRtPFo82QVlrajGgxpMmAlj+ktTGiuTGgc8bnK+efp9Fr0+x3dg1A1XyJucoMc+UZ5suzzFWSer6yWRYqs8xXaswWy9mn3ablYmudIAo5JW8tiduQ1vqPST7cs318z7lk6kHmerwqXuCDK3912tMQQlwnA0WVHFVynGT2yjtPrAZFxLQZ0aBPgwEN+jQZsE6fxnDAxnBAs+bwwup5Pvv8V1jvtnb8tJ1lmixUakmpzrJQqbFYrbNYrbFQrbFYTfpLM3WKbv4GP/vEy2sXATg9f81RUAixg0MZZIQQtycTgxoFahS4e7edntpsxmhaDFmjxzr9rF6P+qw1e6w1+1w6tcETZ59jtdMkiqPLDldy8yzO1Fmq1lmcqXNkZo6ltL80M5f25/b8yaOX0iBzak5WZIR4JSTICCFuWQYqCz737rZT+pbXOPRcossaPVbpcYkuq16PtUs9GlX48svP8KEv/eGO1+OV3DxHZuc5MjvH0ZmkPjIzx9HZeY7W5jk2O898ZTa798/L6xdRSt1298US4kaTICOEEGwNPfezQ7h4Jqk0mh4+l+hmZYUul7wuKytd1ooen3rmS1xsrRNGW1d4LNPk6Mw8x2oLLDdXKbl5/ssnPsiJ+gIn6kscry3KPWWE2CMJMkIIsQcKlX366wxzl+/wfFLFaBr0WaHLRTqs0GE56rDc6HCprlhpNfDCgB/5L/96y8NnimVO1Jc4VV/iRH2Rk3NLnKgvcXJuiZNzR5gtlm/CsxTi8JAgI4QQ+8BAMU+JeUq8mm3XwTwDb+IFvokH+DG+hQu0OU+LC7S50G9zvt/mufAcj3318/S33Wizmi9xcm6JU3NHsvrU/JGs3q+Lk4U4qCTICCHETTYiZIUuJ5nlCBWOUOFNnNi603LyNlaTIedpcZYWF2hxdtji/LkWz0Xn+IOvfO6y63UWKjVOzyeh5vTcUU7PH+WOhaPcsXCMpWpdvn5C3HIkyAghxE12gRYAJ5i54n5q4rqd12y/Y08adBoMOEuTc7Q4S5OXO02Wjyk+89yT/OZn/mDLx89ztsPp+STc3LlwLCmLxzmzdILjtQX5ElJxKEmQEUKIm+xsGmSueg+dq1Ao5igyR5E3cHxzQ/oR9ICI87R5mY0k5ARNXlpucs5Y4RNPf3HL21au5XDHwlHOLJ7gzNIJ7l46yT1HTnJm6SS1UuUVzVOI/SRBRgghbrJzNIFXHmSuxsbkDmrcQW3rhvPJas4lerzEBi/Q4IWwwYvLDZ7RL/P7f/xpgijMdq+Xqty9dJK7jyTh5p6lU9xz9BSn5pZkFUdMnQQZIYS4yc7SwsVkgdLU5qBQLFFmiTJfz6nNDRchJOYsTV6gwfOs83yvwctGxIe+/If8l0/8XrarY9mcWTzO3UdOcd/R09xz5BT3HjnF3UdOkLOv4zu9hLgOEmSEEOImO0uT48xg7O1r5m4aC4M7qXMndf406bdzp/fRaTLkBdZ5lnWeC9d57sI6TwTP8juPfzy7HsdQBqfmj3Df0VPce+Q09x09zX3H7uDeI6couLkpPStxq5IgI4QQN9k5WpzY57eV9sssed7ICd44+Smr1eSTWC/Q4DnWeEav8dzqOs/bF/nwE5/N3qZSSnGyvsT9x+7ggeN38uCJu3jw+F3ctXgcy5TTkbg+1/RvjlLqPuAY8BmtdW9i/B1a6w/t1+SEEOJWdJYmr+fYtKdxQ+WweIBFHpi8K/KF5ILjl9jgmTTgPLu+xrPuRT785GeyOx/nbIf7jt6RBJsTd/Hqk3fz4Im7mCnIzf/E1V01yCilfgj4QZLr4H9JKfXDWuvfTjf/U0CCjBBCXINHeIJ/yodpM+K3eZI3c5K/wKunPa19ZWNyN/PczTzfPh68AB4hz7HOU1ziq8EKzxRDPvTlP+S/fvKD2WNP1pd49ckzPHjiDK8+cYZXnzzDqbkjKHUw35IT03EtKzJ/A3ij1rqnlDoN/IZS6rTW+ufggL7BK4QQB8wjPMGP8TsMCQBoM+LH+B2AWz7M7MTF4lUs8SqWgNfCV5NPUq3S46tc4ius8FTjEk85Z/nglz6F1hqAaqHEa07evaXcc+SkvDV1G7uWf/LG+O0krfVLSqm3kYSZU0iQEUKIa/IzfCQLMWNDAn6Gj9yWQWYnCsUiZRYp882cSQYvwgCfp1nlK6zwlcEKT3kDfumx32IU+EDy1tSDJ87w2lP3pOVu7j96B67tTPHZiJvlWoLMJaXU67TWXwJIV2a+A/hPIP/1CSHEtVimvadxsamAwxs4vnnTvxeTj4g/zzpPssJXgos85Xj89z96lF967LeA5KPhD564izfd+QBvvON+3nTnA9y1eFzelroFXUuQ+StAODmgtQ6Bv6KU+g/7MishhLjFHKXKhR1Cy1GqU5jN4WdhcC8L3MsCf5HXwNPJN46/TJMnuciXw2WedAb86if/Bw9/5P1A8s3i41Dz5jsf4I13PiB3Lb4FXDXIaK3PAyilfg74+3r8RmWy7VP7ODchhLhl/DjfuuUaGYA8Nj/Ot05xVrcWA5Xdyfh/5VXwNYiIeYY1vsgFvtg/zxOtBv/yd34lu+fNmcUTvPmuB3jTXa/izXc+wKuO3ynX2xwye/mn1QU+oJT6Hq11Xyn1Z4F3a62/cZ/mJoQQt4zxdTB/n0eI0Byjyo/zrXJ9zD4zMbifRe5nke/jDXAe+vh8mWW+wHm+eOk8Hx5+ll//w98HoODkeOOd9/Pmu17Fn7jrQd581wPUy1f+ck8xXdccZLTW/1gp9X3AR5VSPtADfnzfZiaEELeY/4X7+SHezz/kbfwD/tS0p3PbKuLwDZzmGzgNgO5oztHicc7zBf88Xxp1+Ln/8etEcXKfm7sWj/MnzjzI1931IH/y3tdxZumEXGtzgFxzkFFKfSvJR7H7wBHgr2mtv7ZfExNCiFvNWZpo4PT2L3EUU6VQnGSWk8wmK2QvJZ8o+zLLfJ5zfOHSeX6//2l+7VPJbdMWKv9/e3ceH9dd3nv882g0kkbbSJZs2ZLlfY9N7DiODdkIDpBSEjANLaG0haTNbW/bS4B7U3N5tTS97b0u3dIFXm2ANLctDbubQOkN1ARCSJzNSfAWx453y7Ysa19HmvndP2ZGkeSxrG3mzJn5vl+veZ2Zo9HRc+LY8+j3e37PbxbXr7ya61eu54aV61lVv0iJjYcmM7X0GeD3nXNPm9k64Gtm9knn3A/TFJuISE45xkWAS3ejlqwTIsgWFg5vqOm6HUe5yG5OsLvzBM8f2cfOF54EoLaiihtWbuCGVeu5adUGVsxbqMQmgyYztfSOEc/3mtnPAd8C3paOwEREcs1xWgGNyPiRYSyllqXU8stsxLU5TtLObo7zbNcJdh/dz7+9GE9sZldWc8PK9dy46hpuXn2Nln2n2ZRLs51zZxPTTSIiMgHHaKWaEFWEvA5FpskwFlLNQqr5JTbgWuOJzbMc55nO4+weMWJTXz2bG1dt4KZV13DT6mtYUDvX4+hzy7TWmDnn+mYqEBGRXHecVo3G5KiRic2H2IBrcxyjlWc4zjNtx9i173m+9uz3AVg0u563r9nIrWs3c/OajVSGyjyO3t+0WF5EJEOO0crmRM2F5DbDWEINS6jhI2zEdTle5wI/5Rg/vXCMbz23i0d+/B0KAwG2LFvHres2c+vazbzN3aBpqElSIiMikgH9DNFEB4uo9joU8YBhw52I72Yzg/1RXuQUP4oe4RWCMdoAACAASURBVKne8/zhN/+BP975JVru+wDhsLo9T4YSGRGRDEguvV5MjdehSBYIEuCtLOKtLOLTp+AcXeyLnlUSMwVKZEREMkArlmQ8c6lgLhVeh+FLBV4HICKSD9RDRiQ9lMiIiGTAcVqp0tJrkRmnREZEJAOO0arRGJE0UCIjIpIB6iEjkh5KZERE0qyfIc7QoURGJA2UyIiIpNmp4aXXSmREZpoSGRGRNDumpdciaZMViYyZPWxmzWa2z+tYRERmWrKHjJrhicSZWaOZPWlmB8xsv5l9fMzXP2Vmzsxqr3StrEhkgEeA27wOQkQkHY5xkTAlVGvptUjSEPAp59waYAvw22a2BuJJDvAu4ORELpQViYxz7ilI/MoiIpJjjtOm0RiREZxzZ51zexLPu4CDQEPiy38F3A+4iVwrKxKZiTCze83sRTN78SK9XocjInJFO9nLZh7kJxzlEM3sZK/XIYlkUm3yczvxuDfVm8xsEbABeM7M3geccc69OtEf4pu9lpxzDwEPAVxt9RPK0kREvLKTvdzPd+hjEIA+Brmf7wCwjXVehiZyRW5wiGhTy3Qv0+Kcu3a8N5hZOfAt4D7i003/k/i00oT5ZkRGRMRPdrBrOIlJ6mOQHezyKCKR7GJmQeJJzFecc98GlgKLgVfN7DgwH9hjZnPHu45vRmRERPykiY5JnRfJJ2ZmwJeBg865vwRwzu0F5ox4z3HgWufcuENDWTEiY2aPAs8CK83stJnd43VMIiLTUU94UudF8sz1wK8A7zCzVxKP90zlQlkxIuOcu8vrGEREZtJ2to6qkQEIEWQ7Wz2MSiQ7OOeeBuwK71k0kWtlRSIjIpJrkgW9n+QxBonSQJjtbFWhr8gMy4qpJRGRXPR+1lJMgF9jE89xn5IYkTTQiIxIltrJXnawiyY6qNdv8750ji66ibCcK3ZZF5EpUiIjkiVGJi5VhOhmgEFiAJyhg0/yGL/Pf9BBnxIbnzhCfLHFcmZ7HIlI7lIiI5IFxjZPa6PvkvcMEqU9cf4MHWqu5gOHuQDAMo3IiKSNEhkRDyVHYc5MobdIsrmaEpnsdYQWKilmDuVehyKSs1TsK+KR5CjMVJKYpDN00MgDbOZB7eOThQ7TwjJmY+OvMhWRaVAiI+KRVC3sp8Lx5lSTkpnscoQLKvQVSTMlMiIZlNwNuZEHrjgSEyRAFSEMqCZE8Ap/XbWPT3Zpp48L9Kg+RiTNVCMjkiFjC3rHk6p52shVTZfb/l37+GSP5IolJTIi6aVERiRDJjKVFCLI57g9ZQHvNtYNn9/MgylHdAowGnlAy7OzwJsrlrT0WiSdNLUkkiHjjZYY8VGYyyUxY21nKyGCl5yP4lQzkyWO0EIxARZQ5XUoIjlNIzIiaZacErrcdFADYZ7jvkldM5nsJKeaCjCiY36Clmd76wgtLKaGgH5fFEkrJTIiaXSlupjp7IY8cqqpkQdSvkc1M945Qgtvod7rMERynn5VEEmj8epiJjOVdCX1hCd1XtKrj0FO0qZCX5EMUCIjkkaXGxExmNHdkFPVzBRSMOXRHpmeY1zEgXrIiGSAEhmRNEj2i7lcXcxMj5RsYx2f43YaCGPEp6yGiPFHPKHOvx44PLz0WiuWRNJNNTIiMyyddTHjGVkz86+8xO/xXS7QA2iTyUw7zAUKMJZQ43UoIjlPIzIiMyxTdTHj+Wt+cslokDr/Zs4RWmikihL9riiSdvpbJjLDrlQX42UMWsWUGfHNIlUfI5IJGpERmWHZsIIoG2LIR8naqEM08xwnVJckkgFKZERmSPJDLNXWAemqi7mcVKuYggS0iimNkrVRyT//biLqriySAUpkRGbA2A8xiE8lQebqYkYau4opSAEhCnk3KzMWQ75JVRuluiSR9FONjMgMSPUh5pja9gMzZeQqphc4yTb+kS/wU/47t3gST65TXZKINzQiIzIDsv1DbBMLuIb5PMhT6iuTJqpLEvGGEhmRGZDtH2I72csBzgFod+w0SVWXlOnaKJF8pERGZBqyqcB3PDvYRT9Do86pfmNmbWMdf8J7hl97URslko9UIyMyRak6+Bpv1sZsZ2vWfIhl+9RXrljFHAD+njt5L1d5HI1IflAiIzJF2Vjgezn1hFOOGmXL1FeuOMB5ANYw1+NIRPKHppZEpshPoxyp6jcKsKyZ+soVBzhHiCALqfY6FJG8oURGZIqyvcB3pLF9ZSopJoZjqTY1nFEHOM8q5hDQP60iGaO/bSKT5JcC37G2sY7nuI9TfJbd3EeYEh7kKa/DyhkOx0HOs4Y6r0MRyStKZMQ3rLKY6m/9IlZZ7FkMqTr4JvlplUolJfw6W/g+h9jHWa/DyQlNdNJBv+pjRDJMiYz4RskdKwltW03J7Ss8iyFVgS+8WeDrhyQm6W42U0KA9/OwmuTNgGSfHo3IiGSWEhnxjdKPrU8cN3gWg58KfK/khxxmCEc/Q2qSNwOSK5ZWK5ERySglMuILVllM0VsbASh6WyNWUeRJHH4q8L2SHexiiNioc2qSN3UHOM9CqinHu6lPkXykREZ8oeSOlbhIFAAXiVJyR+Z3cd7JXnqJXHI+mwt8x5NLo0vZ4ADnNBoj4oGsSGTM7DYzO2RmR8xsu9fxSPYp/dh6ChJFvgWVxRmfXkoW+bbRN+p8FSHfFPiOlUujS17rJcJxWlUfIzIJZvawmTWb2b4x53/XzF4zs/1m9rkrXcfzzr5mFgA+D7wTOA28YGaPO+cOeBuZZFr1Nz5I6BfWpPyaGxi9T1DR9Y3Uxz6b8r193zpA2we/MaOxXa7It4wiXyYxEG+SN3aLhRIKfTm65LWDNONQR1+RSXoE+Dvgn5InzOwW4H3A1c65ATObc6WLZMOIzHXAEefcUedcBPgq8ZuQPNP56V1EXj5LrPvS6RsrLhz3NUCsO0Jkz1k6Pz3zNR65OA0ztkkewLtY4dvEzEsHh7cm0IiMyEQ5554CWsec/i1gh3NuIPGe5itdJxsSmQbg1IjXpxPnJM9Ej7TSsumLdP3Bk8R6B3FDsSt/E+CGYsR6B+n6gydp2fQQ0SNj/15M3zwqU573+zTMyCZ5N7CY5zhFhKjXYfnOQc5RQTGNVHkdiojfrQBuNLPnzOzHZrbpSt/g+dTSRJnZvcC9EO/ZITkq5uh5cDf9332d6q/eSeGKGgrKL79CKdYdYej1Ftp+6ZtE32ib8XB2spcd7KKJzku+5tci38u5l7fyq/wrj7OPO7na63B85QDnWU0dNjy2JeJvBeUhQjetnd5FHqHWzF4cceYh59xDV/iuQmAWsAXYBHzdzJY459xlY51elDPiDNA44vX8xLlRnHMPOeeudc5dW0NpxoITb8RHZx6ie8fTuN5La1MAXO8g3TuepmXTF9OWxIzt4pv8mPJTF9+JuoVlrGA2D/Esjsv+myFjxBJbE2jFksglWpKf24nHlZIYiM/KfNvFPQ/EgNrxviEbEpkXgOVmttjMioAPAY97HJNkAwfRkx2XnWJyQzGiJ9pJ12duqgJfhz+7+E6EYfwGWzjAeTbwF+r2O0GnaKebiOpjRGbGvwG3AJjZCqAIaBnvGzxPZJxzQ8DvAE8AB4GvO+f2exuVZIuRy65dzBHrieBi8cwlncuwd7I35X5K4O8C3ytJ7trcQo+6/U5QcmsCjciITI6ZPQo8C6w0s9Nmdg/wMLAksST7q8CvjTetBFmQyAA4577nnFvhnFvqnPsTr+OR7DCym68bGCLW3EP7PY8Ta+4ZXo6dji6/ySmly/F7ge94/oIfXXJO3X4vbyd7+SSPAfCbfEMJn8gkOOfucs7Nc84FnXPznXNfds5FnHMfcc6tdc5d45z74ZWukxWJjEgqJXesxEVjxHoiDL7URPPaL9D/9f00r/0Cgy81JUZnYjPe5fdyPWMg9wp8x8rFZebpkkx4uxgA4rtfa/RKJPOUyEjWKv3YegrKiuj90h5abn4E1xrvquta+2i5+RF6v7SHgtKiGZ9eGu9DO9cKfMdSt9+JS5XwavRKJPOUyEjWijb30HrXN+n8xBMQHTNFGnV0fuIJWu/6JtHmnhn7mTvZS8FlltA2EM7pJAbi3X5DBEedy/VRqKnS6JVIdvBNHxnJP+13feuK7+n/2n76vzYzteHJqYJoimVQ+fJhnkzUdrCLM3RQSAE7eG/OJ3BTUU84ZUG4Rq9EMksjMiLEk5j72JmyNiaA5fyU0kjJbr9/wzaGiFFLmdchZaXtbKVwzD+h+ZLwimQTJTKS98YbiYF4w7N8SWJG+nnWUEsZ/8jzXoeSlbaxjiXMIpiYjMzFJokifqCpJcl5b24z0EE9YbazdfjDJjkSc7kkBvJ3qqCYQn6Za/gbfsIJ2lhItdchZZUoMZro5ENs4P/wXq/DEclbGpGRnDZym4Fkg7ff5dvM5wEW8Ef8Lt8eN4nJ96mCj3AtBRj/xAteh5J1jtBCNxGuYb7XoYjkNSUyktPG6wkTu8LeBvlWG5PKPCpZxzwe4lltWTDGy4kt4TYokRHxlBIZyWlTXQobIsiDbMvrJAbiI1oHOI8DbVkwxh5OE6aEJdR4HYpIXlMiIzltKvUtGol50w52ESE66pyavsW9zBnW03DZvkMikhlKZCSnbWfrpD5mNBIzmpq+pdZDhEM0s54Gr0MRyXtKZCSnbWMdv8K1E0pmqghpJGYMbVmQ2qs0EcOp0FckCyiRkZz3v/l5/oYP0JD48A0k0prksYEwf8sH2Mf9SmLGSLVlQQmFeb2SC+BlTgOwQSMyIp5THxnJC9tYpyRlCkZuWdCUWML+Tlbk/X/LPZxhEbOYRanXoYjkPSUyIjKukUngL/J/eZkzRIkRyNMBXYfjZU5zPYu9DkVE0NSSiEzCr7KJ03TwJEe8DsUzTXTSTLfqY0SyhBIZEZmwd7OSOsr5J170OhTPqD5GJLsokRGRCQsS4MNs5EkOc4I2r8PxxMucoZgAa5jrdSgighIZEZmkD3MNBRj/kqejMns4zVrmUUTA61BEBCUyIjJJ86hkLfP4hzzbf2kne9nMg7zAKQ7RnBf3LOIHWrUkIpOyk70c5PzwppvJ/ZeAnF2WndxFPbkBaTeRnL9nEb/QiIyITEo+7r+Uahf1XL9nEb9QIiMik5KP+y/l4z2L+IUSGRGZlHzcfykf71nEL5TIiMik5OP+S9vZSvGYksIQwZy+ZxG/UCIjIpOyjXV8jttpIDy8q/htrMrpotdtrOPnWAWAEd9oVDuli2QHrVoSkUkbuf/SnTzCHk7n/P5LbfSxnFqe5Le9DkVERsjdf3VEJCM+yiZO0p7T+y8NMMRznOAGlngdioiMoURGRKbl3ayijgoe4QWvQ0mblzhNP0PcqERGJOsokRGRaQkS4CNs5Ecc4SgXvQ4nLZ7mKAGMLSz0OhQRGUOJjIhM24e5hkIK+Occ3X/pJxxlPQ1UUuJ1KCIyhhIZEZm2Oip4C/V8id05t/9SB/28SpPqY0SylFYtici07WQv+zmb2H0pt/ZfepbjxHCqjxHJUhqREZFp28EuBnJ0/6WfcJQQQa5hvtehiEgKSmREZNpyeS+ipznKFhZSRMDrUEQkBSUyIjJtuboXURMdvMFFTSuJZDElMiIyban2XyrOgf2XnuYYgAp9RbKYp4mMmX3QzPabWczMrvUyFhGZulT7L22kwbeFvjvZy2Ye5JM8RgHGQc57HZJIzjGzTyRygH1m9qiZTam/gdcjMvuADwBPeRyHiEzTNtbxHPdxis/yy1zDS5yhlV6vw5q0nezlfr7DmUR9TwzHdr6bM8vJRbKBmTUA/w241jm3FggAH5rKtTxNZJxzB51zh7yMQURm3q+zhQGGfNkgbwe76GNw1LlcWYElkmUKgZCZFQKlQNNULuL1iMyEmdm9Zvaimb140Ye/5Ynkk+XM5haW8QjPM8CQ1+FMSi6vwBLJsNrk53bicW/yC865M8CfAyeBs0CHc+77U/khaW+IZ2b/CcxN8aXPOOcem+h1nHMPAQ8BXG317gpvFxGP3ctbuYt/5jH28Yus9zqcCasnPDytNPa8SL6Ineik6+5pj0K2OOdS1r+aWTXwPmAx0A58w8w+4pz7l8n+kLSPyDjnbnXOrU3xmHASIyL+cwOLmUcF9/MdX21bsJ2tFI/5HS9E0PcrsESyzK3AMefcBefcIPBt4G1TuZBvppZExF/+jX200MsQMRxvbluQ7cnMNtbxDpYBYEADYT7H7b5dgSWSpU4CW8ys1MwM2AocnMqFPN1rycy2AX8LzAb+3cxecc6928uYRGRm7GAXg5fZtiCbkwKH4zAtbGEh3+SjXocjkpOcc8+Z2TeBPcAQ8DKJ8pHJ8jSRcc7tBHZ6GYOIpIdfi2YP0swRWriHzV6HIpLTnHOfBT473etoaklE0sKv2xY8zj4CGO9htdehiMgEKJERkbTw47YFDsfj7OMGllBDmdfhiMgEKJERkbQYu22BASuozer6mFdp4iTt3MFVXociIhPkaY2MiOS2bawbTlwe5Mf8OT9iH2dZyzyPI0vtcfYRpIDbNK0k4hsakRGRjLibzYQp4a+ydGu1GI7H2c8tLCfMlPauExEPKJERkYyopIR72MwTvMYBznkdziVe5BTn6NK0knjC4ThLp9dh+JKmlkQkY+5hC1/gae7gywwwRD1htrPV07qZnexlB7uGtyXo99neUOJf3QzwNMf4EUd4quYMF7s7aI3soKioyOvQfEWJjIhkzA85zBCOoUSjvGS3X8CTZGYne7mf74za7fr3+Q+KKczqomTxpygx9nKWn3KMp1e1svvwXgajQ5QXh3j7gmvZuu46otHolS8koyiREZGM2cEuhoiNOudlt98d7BqVxHgdj+SWGI7XaOYZjvHc+i6eOfQzOvq6Abiqewn/9V0f5J3rtrB52VqKCuOtCkKhkJch+5ISGRHJmGzr9ptt8Yi/Jbe3eIZjPMsJdpfHp4sAljQ1sO26W7hp9TXcuHIDc8KzPI42dyiREZGMqSc8XIsy9rwXsi0e8ReH4wgtPMtxnuE4uyuaaOlqB6Cxpo53rdzCzas3cuPqDcyfVedxtLlLiYyIZMx2tl5SkxKkwLNuv9vZysfZmdifOy5EMKu7D4t3YjgO0cxuTrCbEzxfeZYLnW0ANFTP5tZVm7lh1XpuXLWBhbXziG/qLOmmREZEMiZZd7KDXTTRQREBCjBuZqkn8axkNjEclRTTxUBWrKKS7NFLhFdoYg+neInTvFR+ntbEVFFjTR1bV1zH9Suv5vqV61kyp0GJi0eUyIhIRo3s9vs6F3gXf8+fsos/5faMx/J3PE05RTzDx6lCRZb5zOE4RXs8YeEULy/sYN+pN4jG4quIltU18p7l1w8nLgtrs7M7dT5SIiMinlnBbO7mOr7Ibj7MRq6mPmM/+ygX+Q77+S2uVxKTh3qI8CpN7OE0ezjNy5XNw9NEZcUhrgmt4hPv+TDXLV3LpqVrmFWuuqlspURGRDz1Cd7Oo+zh/TzMENGMTe98nqcpopDfYEtaf454L0qMw7TwMqd5hTP8rLGT/aePEnPxVgDL6hq5del1XLv0KjYtWcNV85dQGNDHo1/oT0pEPPWfvE4/UQYz0CQv2cW3iQ4ccCOLmU35jP4M8Vay1f+rNPEyZ9i7qpeXj71G90AfAOHScjZWruZ/3P6rbFqyho1LVmu0xeeUyIiIp3awaziJSUpHU7pUXXxf4BQ72aviXp+K4ThBG/s5y17OcvCqCK+eeH24d0thIMC6/uXcdf1tbFy8mmuXrGFp3XwKCrTNYC5RIiMinspUU7pUXXz7GVIXX5+IEOUwF9jPOfZxltdW9LP35BG6+nsBCAYKWd21mPdsuIGrF65g/cIVrFuwjJJgsceRS7opkRERT2WqKZ26+PpHO30c4DwHOMcBznNwQQ+Hmo4TGYonoqVFJayNLuWX3vou3rJwBW9ZsJw1DYspDmqzxXykREZEPJWqSR7A22e4t4y6+GafKDFO0MYBznGQ8xykmddqOjh58dzwe2ZXVrOuYhm33HonVy9YzlsWLmdp3XwCBQEPI5dsokRGRDw1tklePWFKCfJNXqWeMP/KnuHz01nN9EGu5kGeGnVOXXwzw+E4Rxevc4HDXOAQzby+uJ+DZ47RG+kHIFAQYPncRjbNv4q7b3kf6xqXsW7BMurCNR5HL9lOiYyIeG5kkzyAi/RwM5/nz3lyePOA6axmukA3j7KHWsoIEuAcnerimwYxHE108DoXOEILh7nAG0sjHGo6MbzrM8Cs8jBXFS/hozffzlXzl7J2wVJW1S9SPYtMiRIZEck6NZRRRGDEDkhxk1nNNHKpdZAAMWJ8j//CGrR533RFiHKcVo7Qwhu0cJgW3ljYz+GzJ4dHWABqysOsLlzMnVtuZXX9IlY1LGZV/SJmV1Z7GL3kGiUyIpKVLtCd8vxEinPHLrWOECVIgEM0K5GZhFZ6eYMW3uDi8PFoXS/HLjQNt+6H+IaJK8oX8ms3vZcV9QtZVb+IlfMWUlNR5WH0ki+UyIhIVrpccW4VITbz4Lh1M6mWWg8S1VLrFPoY5ARtHOXi8OP40kEOnztFW0/n8PuKCoMsrZvPmrlLeP+mt7Ny3kKWz1vIsrpGKkKlHt6B5DslMiKSlS63mqmdPtqId2lN1s28wEl2cXg4uUmVAEH+LrXupJ+TtHGcNk7QyknaOL3aONp8mtOtzTj35iReXXgWywsX8L5rb2bFvIUsn9vIsrkLWFg7VyuFJCspkRGRrDR2NdM8wlykm4EUXYD/mRdHFQVfTq4utR4ixlk6OUkbJ2jjFG2cpJ3Ti6Mcu9BEa/fo/yY15WEWDzTwthVXs2ROA8vmLmDZ3PksmTOfylCZR3chMjVKZEQka41dzdTIAynfN7YoGMDGnPfzUusYjma6OU07J2njFO2cop2m1caJC2c509bMUPTNBK8wEGD+rDoWlszjjo03sXhOA4tn17N4TgOLZtcrWZGcokRGRHxjvGmjsRzQQHhGetCk2yBRztHFGTo4TTtn6OAU7ZxbE+Bky1lOtzYPd7VNqgvPYkFkHpuWXsUv1G5l0ex6Fs2ex6LZ9TRUz9buzZI39H+6iPhGqrqZsSMvSQ2EeY77Mhbb5cRwtNLLOTppopMmOmiikzN0cH6ZceriOc62XyTmYqO+ry48i8b+uaxftJLbN95EY81cFtXOY0HtXBbUziNUpJ4rIqBERkR8JFUX4K0s5xu8Oiq5ydQ0Uh+DnKeL83Rxji7O0Zk4dtGyPEBT2wXOtrdcMpoSDBTSMGsOjYV13LxmI/Nn1dFYU8f8mjoaZ9XRWFun5nAiE6RERkR8ZWzdDMAmFoxKbqYzjeRw9BChmW7O00Uz3VwY8bxldZBz7S2ca784qlttUmlRCfOqa5lXUMt1y66ivno2DdVzqK+eTX11LfNr6phdUU1BQcGU4hOR0ZTIiIjvpUpuxupjkAt0c4EeLtKTeB5/NNND67IAzZ1tNHe00jPQd8n3FxUGmRuuYW6khpX1i7h5zUbmVtUyr6qGunAN9dWzmVddSzhUjpml61ZFZAwlMiLiSxGiXKSHNnq5SC8tiQQl/uiNf21pIc2dbbR0ttGdIjkBqC6rpC48iznBWVy7ZDVzKmdRF65hbiJBqQvHX1eXVShBEclCvkxkjtHKJ3mMakJUE6KKEFWUjngef5QSxNA/PCLZLDmV00E/7fQNN7xrpzdxjL9upZf2JYVc7O7gYlc7nX09Ka8XKAhQWxFmdkU1NUXlbFq6hjmVs6itqGJO5SzmhKuprahmTriaOZWzKCoMZviORWQm+TKRGSqCZ8qbaO3uHLVB2ViFgQDVZZVUl1VQVVpB2RsDhAkRpoQqSggTopISKikhnDgmHxUUU4jmsEUmYogYXfTTQT9dDNCZeB4/9tFJP+2Jcx300b20mPaeLtp7u2jr6RzVA2Ws0qISZpVXMqs8zKySUhbNrqe2IkxtRRWzyquorQhTU1HFnMpqaiuqqCqtUP2JSB7xNJExsz8DbgciwBvAx5xz7Vf6vpXzFvHjz34RgP7BAVq7O2nr6aS1u5P2nvg/jG2JY3tvF23diX8wFw5xrPci7b1ddPR2j2rLnUpZcYjKUFn8UVpO6Rv9VFA8/CgfcyyjmHKKho/lFFNGEcX+zBclxzkcA0TpJUI3A3QToWfEsYsBuscck4+eJcV09nXT1ddLZ19PypqSkcyMcGk5VaXxXyqqikM0VM+huryC6rJKqkorqC5LPC+riCcuZWGqyyu0ekckR5nZbcBfAwHgS865HVO5jtefsD8APu2cGzKzPwU+DfzeZC5QEixOrAaYPakfHIvF6Ozvob2ni86+Hjr7euhIJDidvT109HXT2dtNR+JrnX3ddC6O0dTXTXd/M119PZedcx8rGCikrCREeXGIsuIQZSUhSo71U0YRpRQRIkgpQUopGj6GRhzHPkoopGTEUSNHuWuIGP0M0s/Q8LGPwUsevUSGj70jjslzPUToX1xCT38fPQN9dA/00dPfx2B0aEJxlBeHqAiVUREqpTJUTkWolIZZc6gMlRFOJPrhUDnh0nIqS8sIl1YQDiWfl1NZUqZREhEZZmYB4PPAO4HTwAtm9rhz7sBkr+VpIuOc+/6Il7uBOzP1swsKCoZ/O5yqaCxKd38fXf09dPX10tXXQ1d/L939vXT399Hd3zv8uifxwZH8AOldUcS5/j56B9rpHeiPf32gn2js8kPsl1MYCBAKFlNSVEwoWExxsIiSYBHFwSIKj/ZQTOHwo4hA4lFIceKYPBdMPIooJEjB8OvCxPORx8Lh45uPwPDDRj0vGHHOsMQ5S7xKfw2TwxFLPKI4HI4oMaI4Yolj/PXo50OjHtHh54OJ58ljhChDRBkkRoQhBokySJSBxDHCEAOJY4QoAyOOAwwxtKSMgcEI/YMRBgYj9A0O0B8ZoG9wYNwpl8sJFAQoLw5RWlySeISoKKmkrlMbewAABlhJREFUpjjEgpp5lBWXUFYST6rLS0qpKCmlvKSU8pI3XyeTloqSMspLQtosUERm2nXAEefcUQAz+yrwPsBficwYdwNf8zqIyQgUBAiXxn8LnQnOOSJDg/QM9NMb6aM/EqE30k9fpJ+egfixb2Bg1AddXyT+SL6ODA3SPxihPzIQ/2BcEaR7MELr0CD9g30MDEWIDA4yMDTIwGCESHRwSh+WM6nACjAzCswwiyc3ZgyvEEkmOyNXjCSnBV2ip6tzDucSSYuLxZ87d0m31EwrDAQoCgQpDhZRXBikKBikuDCeaBYVFlESLKc0WEQoPGs4AS0qDA4npqVFJZQUFY1KVEuLSwgVlVBaXBw/Jt5TWhSirLiEosKgVteISLZrAE6NeH0a2DyVC6U9kTGz/wTmpvjSZ5xzjyXe8xlgCPjKONe5F7g38XIgfPdN+2Y6Vh+pBVq8DmKmxFwMHEwwnfLVvQ9FowxFo+MWpU+Sr+4/DXT/uv/cvv+7L/uVlRmMgp9x9on5PFA7zcuUmNmLI14/5Jx7aJrXvETaExnn3K3jfd3MPgq8F9jqxqm+Tdz8Q4nvedE5d+1Mxukn+Xz/+XzvoPvX/ev+8/X+xyQEaeecuy3NP+IM0Dji9fzEuUnztPouUbF8P3CHc67Xy1hEREQkY14AlpvZYjMrAj4EPD6VC3ldI/N3QDHwg8Sc/m7n3G96G5KIiIikU2K18u8ATxBffv2wc27/VK7l9aqlZVP81hmfY/OZfL7/fL530P3r/vNbPt9/zt27c+57wPemex27UlM4ERERkWylDlUiIiLiW75NZMzsz8zsNTP7mZntNLMqr2PKFDP7oJntN7OYmeVNBb+Z3WZmh8zsiJlt9zqeTDKzh82s2czysu2AmTWa2ZNmdiDx//7HvY4pU8ysxMyeN7NXE/f+gNcxecHMAmb2spl91+tYMs3MjpvZXjN7JdOrl/zAt4kM8e0N1jrn3gK8Tnx7g3yxD/gA8JTXgWTKiHbWPwesAe4yszXeRpVRjwDpXg6ZzYaATznn1gBbgN/Ooz//AeAdzrmrgfXAbWa2xeOYvPBx4KDXQXjoFufc+nxdfj4e3yYyzrnvO+eSG8XsJr4GPS845w465w55HUeGDbezds5FgGQ767zgnHsKaPU6Dq8458465/YknncR/0Br8DaqzHBx3YmXwcQjr4obzWw+8PPAl7yORbKPbxOZMe4G/sPrICStUrWzzosPMhnNzBYBG4DnvI0kcxLTKq8AzcAPnHN5c+8JDxLvOebtniPeccD3zeylRJd7GcHrPjLjmqntDfxoIvcukm/MrBz4FnCfc67T63gyxTkXBdYnagF3mtla51xe1EuZ2XuBZufcS2b2dq/j8cgNzrkzZjaHeN+11xKjtEKWJzIztb2BH13p3vPQjLWzFn8ysyDxJOYrzrlvex2PF5xz7Wb2JPF6qbxIZIDrgTvM7D1ACVBpZv/inPuIx3FljHPuTOLYbGY7iU+1K5FJ8O3UkrY3yDsz1s5a/Mfirb+/DBx0zv2l1/FkkpnNTq7KNLMQ8E7gNW+jyhzn3Kedc/Odc4uI/73/YT4lMWZWZmYVyefAu8ifJHZCfJvIEN/eoIL4MNsrZvb3XgeUKWa2zcxOA28F/t3MnvA6pnRLFHYn21kfBL4+1XbWfmRmjwLPAivN7LSZ3eN1TBl2PfArwDsSf99fSfyGng/mAU+a2c+IJ/Q/cM7l3RLkPFYHPG1mrwLPA//unPt/HseUVdTZV0RERHzLzyMyIiIikueUyIiIiIhvKZERERER31IiIyIiIr6lREZERER8S4mMiIiI+JYSGREREfEtJTIiMilm9rCZNZuZuouKiOeUyIjIZD1CfK8fERHPKZERkZTM7Ekze2fi+R+b2d8CJHbdbfU0OBGRhKze/VpEPPVZ4I/MbA6wAbjD43hERC6hREZEUnLOPZXYdfqTwNudc1GvYxIRGUtTSyKSkpmtI77zcsQ51+V1PCIiqSiREZFLmNk84CvA+4BuM1Nxr4hkJSUyIjKKmZUC3wY+5Zw7CPwv4vUyya8/CjwLrDSz02Z2jzeRioiAOee8jkFERERkSjQiIyIiIr6lREZERER8S4mMiIiI+JYSGREREfEtJTIiIiLiW0pkRERExLeUyIiIiIhvKZERERER3/r/PTV+VCLhK2wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "w = paddle.to_tensor([0.2, 2])\n",
    "model = OptimizedFunction(w)\n",
    "opt = Momentum(init_lr=0.01, model=model, rho=0.9)\n",
    "train_and_plot_f(model, opt, epoch=50, fig_name='opti-vis-para4.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，在模型训练初期，梯度方向比较一致，参数更新幅度逐渐增大，起加速作用；在迭代后期，参数更新幅度减小，在收敛值附近振荡。\n",
    "\n",
    "**简单拟合实验**  训练单层线性网络，进行简单的拟合实验。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAEICAYAAAB74HFBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcZXV95//X+y619k4XTS9AswkiKmC7O+qAGiQqxjGj/NSQxF/I5JdFJ2aMmvkl+JssJhM1mck2qIwkYXBBjYzRBCIYhxkDNNisrYIsTUMv1Q29dy333s/vj3Oqu2iqurZ76yz1fj4e93HvPffcOp9bXf2td33P93y/igjMzMzMbHYqWRdgZmZmVmQOU2ZmZmZz4DBlZmZmNgcOU2ZmZmZz4DBlZmZmNgcOU2ZmZmZz4DBVMpIekPT6DI9/iqQDkqod+Noh6cx2f10zs2NJWp+2ObVp7v95Sb/b6bosnxymSiYiXhAR3wGQdJWkv+3k8SQ9JukN446/JSIWRUSzk8c1MzPLC4cpm9R0/yIzMzNbyBymSmasp0jSJcDHgHelp93uSV9fKulzkrZJelLS746dkpP0s5L+t6RPS9oNXCXpDEm3SNotaZek6yQtS/f/G+AU4H+mx/jwsV3jktZIulHS05IelvQL42q9StKXJP21pP3pKcoN0/ycS9P3DUp6XNJ/lFRJXztT0j9L2pvW/MV0u9LPtlPSPkn3STqvbd98M+uYtC35Svp//lFJvzbutask3SDpi2lbcrekF497/fmSviNpT9rOvG3ca72SPpm2I3sl3Sapd9yh3yNpS9qW/NYM6v2FtM17Om0D16TbJ22HJF0q6cH0Mzwp6Tfm9E2zeeMwVVIR8Q/A7wNfTE+7jTUsnwcawJnABcCbgP973FtfDjwCrAJ+DxDwB8Aa4PnAycBV6THeB2wB3poe448mKOULwNb0/e8Efl/SReNef1u6zzLgRuDPpvkR/yuwFDgdeB3wM8DPpa/9J+AmYDmwLt2X9LO+Fnhe+t5/C+ye5vHMLCPpH0r/E7gHWAtcDHxQ0k+M2+0y4MvACuB/AH8nqS6pnr73JuBE4FeB6ySdnb7vj4GXAK9K3/thoDXu674GODs95m9Lev406r2IpN38t8Bq4HGSdg6O3w59DvjFiFgMnAfcMtWxLB8cphYQSauAS4EPRsTBiNgJfBp497jdnoqI/xoRjYg4HBEPR8TNETEcEYPAp0jCy3SOdzLwauA3I2IoIjYBnyUJPmNui4hvpmOs/gZ48QRf6tivW01r/mhE7I+Ix4BPAu9LdxkFTgXWpMe9bdz2xcA5gCJic0Rsm85nMbNMvRQYiIj/LyJGIuIR4DM8u+26KyJuiIhRknaqB3hFelsEfCJ97y3AN4DL05D288AHIuLJiGhGxP+JiOFxX/fjaVt4D0mYm7KNAt4DXBMRd6df66PAKyWt5/jt0ChwrqQlEfFMRNw94++UZcJhamE5FagD29Lu7j3AfyP5a23ME+PfIGmVpC+kXc77gL8FVk7zeGuApyNi/7htj5P8ZTlm+7jHh4CeaYzVWpl+jscn+bofJulRuyPt0v95gLQR/TPgz4Gdkq6WtGSan8XMsnMqsGas3Urbro+R9KCPOdJ2RUSLoz3ia4An0m1jxtqLlSSh68fHOfaxbdSiadS7hnHtU0QcIOl9WjtFO/RvSP7gfTwdqvDKaRzLcsBhqtzimOdPAMPAyohYlt6WRMQLjvOe30+3vTAilgDvJQkqk+0/3lPACkmLx207BXhyJh9iArs42vv0nK8bEdsj4hciYg3wi8BfKJ1SISL+S0S8BDiXpJv9P8yxFjPrvCeAR8e1W8siYnFEXDpun5PHHqQ9TutI2qCngJPHxlSmxtqLXcAQcEab632Kce2TpH7gBI62URO2QxFxZ0RcRvIH7t8BX2pzXdYhDlPltgNYP9aIpF3JNwGflLREUkXJAPPjnbZbDBwA9kpay3PDxw6ScUvPERFPAP8H+ANJPZJeBLyfpHdr1tJTgl8Cfk/SYkmnAr8+9nUl/bSkdenuz5AEvpakl0p6eTqG4iBJI9p67hHMLGfuAPZL+s10wHhV0nmSXjpun5dIekfas/1Bkj8c/wW4naRH6cPpGKrXA28FvpD2Vl0DfCod4F6V9EpJ3XOs93rg5ySdn36t3wduj4jHJmuHJHVJeo+kpempyn24fSoMh6ly+3J6v1vS2Ln3nwG6gAdJgsYNJAMkJ/Nx4EJgL/D3wFePef0PgP+Ydr1PdOXJ5cB6kr/Uvgb8TkT808w/ynP8KklD9AhwG8mA02vS114K3C7pAMmg9g+kYyyWkIyzeIakC3438J/bUIuZdVD6B9RbgPOBR0l6lD5LMoB7zNeBd5H8/34f8I6IGI2IEZLw9Ob0fX8B/ExE/CB9328A9wF3Ak8Df8gcfzembdz/C3wF2EbS8zU2vut47dD7gMfSIRX/jmTslRWAIo53lsbMzCzfJF0FnBkR7826FluY3DNlZmZmNgcOU2ZmZmZz4NN8ZmZmZnPgnikzMzOzOZjXhWxXrlwZ69evn89DmlnG7rrrrl0RMZB1HXPl9sts4Zlu+zWvYWr9+vVs3LhxPg9pZhmT9PjUe+Wf2y+zhWe67ZdP85mZmZnNgcOUmZmZ2Rw4TJmZmZnNwZRhKl1T7Q5J90h6QNLH0+2fl/SopE3p7fzOl2tmZmaWL9MZgD4MXBQRB9KFGW+T9K30tf8QETd0rjwzMzOzfJsyTEUyq+eB9Gk9vXmmTzMzMzOmOWZKUlXSJmAncHNE3J6+9HuS7pX0aUndk7z3SkkbJW0cHBxsU9lmZmZm+TCtMBURzYg4H1gHvEzSecBHgXOAlwIrgN+c5L1XR8SGiNgwMDC9efsaD+9m32/fQmPLnmntb2aWJ8P3PcHolt1Zl2Fm82RGV/NFxB7gVuCSiNgWiWHgvwMva1dRjUeeYf9/+i7Nrfva9SXNzObNyINP0XCYMlswpnM134CkZenjXuCNwA8krU63CXg7cH/7qlJy3/LQLDMrHvXUiaHRrMsws3kynav5VgPXSqqShK8vRcQ3JN0iaQAQsAn4d22rSg5TZlZcDlNmC8t0rua7F7hggu0XdaQiQGM9U85SZlZA6q7T2j+UdRlmNk/yOQN6mqXcM2VmRVTpqdMads+U2UKRzzB1pGfKYcrMikc9dRhpEs1W1qWY2TzIdZgK90yZWQGpJxlBEcONjCsxs/mQzzDlAehmVmDqqQN4ELrZApHPMOUB6GZWYOp2mDJbSHIZpuQB6GZWYO6ZMltYchmmPADdzNpF0jWSdkp6zsTCkj4kKSStbOsxx8KUr+gzWxByHaY8AN3M2uDzwCXHbpR0MvAmYEu7Dzh2mq/lnimzBSGfYcoD0M2sTSLiu8DTE7z0aeDDdGB0pipC3TWf5jNbIPIZpjwA3cw6SNJlwJMRcc8U+10paaOkjYODgzM7RreXlDFbKPIZpjwA3cw6RFIf8DHgt6faNyKujogNEbFhYGBgZsfx+nxmC0Yuw5Q8AN3MOucM4DTgHkmPAeuAuyWd1M6DqKfuAehmC8SUCx1nouIxU2bWGRFxH3Di2PM0UG2IiF3tPI566sTgvnZ+STPLqVz2TI0NQPfVfGY2V5KuB74HnC1pq6T3z8txe+rEUINwD7tZ6eW7Z8ptkJnNUURcPsXr6ztxXHXXkqEKI03ozmdTa2btkdOeqfTePVNmVlBjE3d6rimz8stnmPIAdDMrOM+CbrZw5DJMyQPQzazgKl6fz2zByGWY8gzoZlZ0XuzYbOHIZ5gaW5vPWcrMCmpsfT6HKbPymzJMSeqRdIekeyQ9IOnj6fbTJN0u6WFJX5TU1baqPADdzApO9SpUKw5TZgvAdHqmhoGLIuLFwPnAJZJeAfwh8OmIOBN4Bmjf3C0egG5mJeBZ0M0WhinDVCQOpE/r6S2Ai4Ab0u3XAm9vV1EegG5mZeD1+cwWhmmNmZJUlbQJ2AncDPwY2BMRjXSXrcDaSd4781XXPQDdzEpAPXVaQ42pdzSzQptWmIqIZkScT7Ig6MuAc6Z7gFmtuu4Z0M2sBCrdNfdMmS0AM7qaLyL2ALcCrwSWSRpbI2Ed8GTbqhrLUu6ZMrMC85gps4VhOlfzDUhalj7uBd4IbCYJVe9Md7sC+Hr7qvIAdDMrPvXUYbRJNFtZl2JmHTSd1TdXA9dKqpKEry9FxDckPQh8QdLvAt8HPte2qjwA3cxKYPzEnervzrgaM+uUKcNURNwLXDDB9kdIxk+1nwegm1kJPGviTocps9LK5Qzo8gB0MysBLyljtjDkMkx5BnQzK4MjYcqD0M1KLZ9h6sjafA5TZlZcY2Gq5Z4ps1LLdZhyz5SZFZm6aiCf5jMru3yGKQ9AN7M2kXSNpJ2S7h+37T9L+oGkeyV9bWz6l7YfuyLU5Yk7zcoul2HKA9DNrI0+D1xyzLabgfMi4kXAj4CPdurgXp/PrPxyGaY8AN3M2iUivgs8fcy2m8atLfovJKs4dEQyC7rX5zMrs3yGKc+Abmbz5+eBb030wqwWaj/2a7hnyqz0ch2mvDafmXWSpN8CGsB1E70+q4Xajz2Gw5RZ6U1nOZn55wHoZtZhkn4WeAtwcXRwHhZ1J4sdRwQaa9vMrFTyGaY8AN3MOkjSJcCHgddFxKGOHqunDgEx3Dgy75SZlUsuT/PJPVNm1iaSrge+B5wtaauk9wN/BiwGbpa0SdJfder4Fc+CblZ6+eyZguSKPg9AN7M5iojLJ9j8ufk6/rPW51s6X0c1s/mUy54pIDnV554pMys4L3ZsVn75DVOSr+Yzs8JTt8OUWdnlN0xV5AHoZlZ46klGU3jiTrPyym+YEj7NZ2aFp1oVahX3TJmVWG7DlCryAHQzKwVP3GlWbrkNUx6AbmZloZ46LYcps9KaMkxJOlnSrZIelPSApA+k26+S9GQ6R8smSZe2tTIPQDezkqh0u2fKrMymM89UA/hQRNwtaTFwl6Sb09c+HRF/3JHKKqK5ZS/NXQepruzvyCHMzOaDeurE3o5OtG5mGZqyZyoitkXE3enj/cBmYG2nC0Mw9JXNbD+pM1nNzGy+eMyUWbnNaMyUpPXABcDt6aZfkXSvpGskLZ/kPVdK2ihp4+Dg4AwqS5eUafpUn5kVm3rq0GgRjWbWpZhZB0w7TElaBHwF+GBE7AP+EjgDOB/YBnxyovdFxNURsSEiNgwMDEy7MFW8urqZlYNnQTcrt2mFKUl1kiB1XUR8FSAidkREMyJawGeAl7W1MjlMmVk5eBZ0s3KbztV8IlkUdHNEfGrc9tXjdvsp4P72VuYwZWblcKRnyrOgm5XSdK7mezXwPuA+SZvSbR8DLpd0PsmiL48Bv9jWypylzKwkxpaU8VxTZuU0ZZiKiNuYONp8s/3ljOOeKTMriYrHTJmVWn5nQPdFfGZWFl01kMOUWVnlN0yN+hJiMysHSZ5ryqzEchumYsRhyszKQ15Sxqy08humRltZl2Bm1jbqqRPDDlNmZZTbMOXTfGbWDukKDTsl3T9u2wpJN0t6KL2fcAWHttbh03xmpZXfMOVlZMysPT4PXHLMto8A346Is4Bvp887ymHKrLzyG6bMzNogIr4LPH3M5suAa9PH1wJv73QdyWm+BtHyH4pmZeMwZWYL0aqI2JY+3g6smminWS/UPoHK2JIyI+6dMisbhykzW9AiIphkZrvZLtQ+kbFZ0H2qz6x8HKbMbCHaMba+aHq/s9MHPLI+35DX5zMrG4cpM1uIbgSuSB9fAXy90weUl5QxKy2HKTMrNUnXA98Dzpa0VdL7gU8Ab5T0EPCG9Hln63CYMiutKRc6NjMrsoi4fJKXLp7POtTtMGVWVvnvmaoo6wrMzOZM1QrUqw5TZiWU+zClXneemVk5qKdOy0vKmJVO7sNUNLxGn5mVQ8WzoJuVUu7DlJeVMbOy8JIyZuWU2zB10lMfov+XNoB7psysJNTtMGVWRrkNU9XVi6mctAiAaDlQmVnxHVmfL9zjblYmuQ1TkF79Au6dMrNSUE8dmi23aWYlM2WYknSypFslPSjpAUkfSLevkHSzpIfS++Vtr66WludxU2ZWAl6fz6ycptMz1QA+FBHnAq8AflnSucBHgG9HxFnAt9Pn7VVN5pjyFX1mVgaeBd2snKYMUxGxLSLuTh/vBzYDa4HLgGvT3a4F3t7u4o6c5ms6TJlZ8Y3Ngt5ymDIrlRmNmZK0HrgAuB1YFRHb0pe2A6smec+VkjZK2jg4ODiz6tLTfO6ZMrMyqIz1THniTrNSmXaYkrQI+ArwwYjYN/61SC5NmXBgU0RcHREbImLDwMDAzKpLT/N5zJSZlYFP85mV07TClKQ6SZC6LiK+mm7eIWl1+vpqYGe7i1PNp/nMrETqVajIYcqsZKZzNZ+AzwGbI+JT4166EbgifXwF8PW2V1f1aT4zKw9JngXdrISm0zP1auB9wEWSNqW3S4FPAG+U9BDwhvR5e6U9U89c8Xdt/9JmZlnwLOhm5VObaoeIuA3QJC9f3N5ynq25ZS8AI995rJOHMTObN2OzoJtZeeR6BvT6i5ILBKtnrsi4EjOz9vBpPrPymbJnKku9l51D7XknUDtnZdalmJm1hXpqnmfKrGRy3TMFoCXdxGgz6zLMrIQk/ft0maz7JV0vqafTx6z01GGk4QXczUok/2GqXoFRNzpm1l6S1gK/BmyIiPOAKvDujh+3e2yuKY+bMiuL3Icp6lWG/+kRdv/0l7KuxMzKpwb0SqoBfcBTnT6gPAu6WenkPkypnpQ4dMODGVdiZmUSEU8CfwxsAbYBeyPipvH7zGk5rEl4FnSz8sl9mKJezboCMyshSctJFmw/DVgD9Et67/h95rQc1mTHdZgyK53ch6mxnikzszZ7A/BoRAxGxCjwVeBVnT6ow5RZ+eQ/qdTyX6KZFdIW4BWS+tJlsy4GNnf6oOpOZqTxmCmz8sh/UomsCzCzMoqI24EbgLuB+0jaw6s7fVxVKqivi+aeQ50+lJnNk1xP2glAy2nKzDojIn4H+J35Pm511VKaO/YSESSdYmZWZPnvmXKYMrOSqa1aQhwaIQ4MZV2KmbVB7sNUhMOUmZVL9aSlADS278u4EjNrh9yHKfdMmVnZVJb1QVeN5o69WZdiZm2Q/zDlLGVmJSOJ2qolNHe4Z8qsDPIfptwzZWYlVF21lNa+w7QOjWRdipnNkcOUmVkGaum4KZ/qMyu+/IcpD0A3sxKqnNAPtYpP9ZmVQO7DVLhnysxKSJUK1YElNNwzZVZ4U4YpSddI2inp/nHbrpL0pKRN6e3SjlXoLGVmJVU7aQmtpw8Sw42sSzGzOZhOz9TngUsm2P7piDg/vX2zvWWN454pMyup6qp0vqmdPtVnVmRThqmI+C7w9DzUMjGHKTMrqerAYqjIg9DNCm4uY6Z+RdK96WnA5ZPtJOlKSRslbRwcHJz5UTwA3cxKSrUq1RMWeRC6WcHNNkz9JXAGcD6wDfjkZDtGxNURsSEiNgwMDMz4QP2/+vJZlmhmln/VVUtp7tpPNJpZl2JmszSrMBUROyKiGREt4DPAy9pb1lF97zqP/l97OVra3alDmJllpnrSUmgFzcH9WZdiZrM0qzAlafW4pz8F3D/Zvu2gegUarU4ewswsE7UTlwD4VJ9ZgdWm2kHS9cDrgZWStgK/A7xe0vkkExc8BvxiB2uEWoVwmDKzElJ3jcqKfho79uL+d7NimjJMRcTlE2z+XAdqmZRqFRh1mDKzcqquWsLoQzuIVgtVcj+Xspkdoxj/a+tVaAWNh3ZnXYmZWdvVVi2FRovW7oNZl2Jms1CMMKXkbudLr862DjOzDjgyeafnmzIrpEKEqdb2AwDE3uGMKzGzMpG0TNINkn4gabOkV2ZRR6Wvi8riHg9CNyuoKcdM5UHj8eSvNS3qyrgSMyuZPwX+ISLeKakL6MuqkOpJS2ls2U1EICmrMsxsFgrRM1VZnISo6slLMq7EzMpC0lLgtaQX1ETESETsyaqe6qqlxHCD1p5DWZVgZrNUiDC17C9+EoDq6ZOuWmNmNlOnAYPAf5f0fUmfldQ/foc5L4c1A7VVY/NNedyUWdEUIkxVVvTR9aqTYaiRdSlmVh414ELgLyPiAuAg8JHxO8x1OayZ0OIe1NdFw+OmzAqnEGEKQD01wmHKzNpnK7A1Im5Pn99AEq4yIYnqqiU0t+8lvMC7WaEUJkzRWyMOO0yZWXtExHbgCUlnp5suBh7MsCRqq5YSh0aIA75y2axICnE1H7hnysw64leB69Ir+R4Bfi7LYsbPN9W1uCfLUsxsBhymzGzBiohNwIas6xhTWd4HXTWa2/fCmauyLsfMpqkwp/nUW3eYMrNSk0Rt1RJP3mlWMMUJUz014vBo1mWYmXVUddVSWvsO0zo8knUpZjZNhQpTnhrBzMquepLnmzIrmuKEqd5kzJQvGTazMquesAi6aow+uivrUsxsmooTpnpqEMBIM+tSzMw6RpUKXc9bRePxXbQODGVdjplNQ2HCFD3JhYcehG5mZdf1/DUAjGzelnElZjYdhQlT6q0DDlNmVn6VRT3UTl3JyI+2E6PujTfLu+KEqbGeKV/RZ2YLQNe5a2GkweiPd2RdiplNoXhhyj1TZrYAVE9cTGXlIkYefMoX3pjl3JRhStI1knZKun/cthWSbpb0UHq/vLNlJlfzgcOUmS0Mkug+dy2tvYdpbH0m63LM7Dim0zP1eeCSY7Z9BPh2RJwFfDt93lFHT/M5TJnZwlBbvxL1dTHy4JNZl2JmxzFlmIqI7wJPH7P5MuDa9PG1wNvbXNdzjIUpT9xpZguFqhW6zllN86k9NJ85mHU5ZjaJ2Y6ZWhURY9fsbgcmXZFT0pWSNkraODg4OMvD+Wo+M1uY6mevhmqFkQefyroUM5vEnAegRzIyctLRkRFxdURsiIgNAwMDsz6Or+Yzs4Wo0lOnfsaJjP54J60ht39meTTbMLVD0mqA9H5n+0qahK/mM7MFqusFa6DZYvSHnsTTLI9mG6ZuBK5IH18BfL095UzOV/OZ2UJVXdZPdc0yRjZvI5qtrMsxs2NMZ2qE64HvAWdL2irp/cAngDdKegh4Q/q8o3w1n5ktZF0vWEscHqHxmBdANsub2lQ7RMTlk7x0cZtrOS5P2mlmC1lt7XIqS3sZfuBJaqcPICnrkswsVZwZ0NOr+Tw1gpm1k6SqpO9L+kbWtRyPJLrOXUtr9wGaO/dlXY6ZjVOYMEWtAhX5aj4za7cPAJuzLmI66mecCF01Rh7wJJ5meVKYMCUJ9dR8ms/M2kbSOuAngc9mXct0qF6l6+yTaGzZTWv/UNblmFmqMGEKkiv6HKbMrI3+BPgwMOElcu2adLidus5ZAxJDdzziBZDNcqJQYYqemq/mM7O2kPQWYGdE3DXZPu2adLidKou66b5wPY0tuxn94fasyzEzChamfJrPzNro1cDbJD0GfAG4SNLfZlvS9HSdt5bq2uUM3fGI1+wzy4FihaneusOUmbVFRHw0ItZFxHrg3cAtEfHejMuaFkn0/qvnoXqVw9/5AdFoZl2S2YJWrDDVU/PVfGZmQKW3i97Xnk1rzyGG7ngk63LMFrTihSn3TJlZm0XEdyLiLVnXMVO1tcvpOm8doz/czqhnRjfLTLHCVG/Nk3aamY3TfeGpVFYu4vD/fojWAU+XYJaFYoUpX81nZvYsqlboe9050AoO//MPiZanSzCbb8ULU+6ZMjN7lsqSXnpfdSbNnfsYvmdL1uWYLTiFClP4aj4zswnVzziR+hknMnLPFhrb92RdjtmCUqgw5av5zMwm1/PKM6gs6uHwP/+Q1pDbSrP5Urww5Z4pM7MJqV6j9/XnEEOjHPrWvbQODWddktmCUKww5bX5zMyOq7pyMX1vPI/WgWEO/v09NPcezroks9IrVJiqLO6G4Satvb7818xsMrU1y+h/8wthtMmhb95Dc/eBrEsyK7VChamu154KwPBNP864EjOzfKuuXEzfpS+GaoWD37rXg9LNOqhYYeqV6wAYfWBnxpWYmeVfdVkf/Ze+mEpfF4duup/Rx3dnXZJZKc0pTEl6TNJ9kjZJ2tiuoiY9Xq2ajJs66KtUzMymo7Kom75LX0xl+SIO3/ogIw9tz7oks9JpR8/Uv46I8yNiQxu+1pTU30UcHJmPQ5mZlUKlp07/JS+kunoZQ7c9xPB9W7MuyaxUCnWaD0D9dVrumTIzmxHVq/S94QXU1q9keOOjHPrODzwXlVmbzDVMBXCTpLskXTnRDpKulLRR0sbBwcE5Hg7UVycOuQEwM5spVSv0vu4cui84lcbjuzj4tbsYfXSQCK/nZzYXcw1Tr4mIC4E3A78s6bXH7hARV0fEhojYMDAwMMfD+TSfmdlcqCK6zz+F/rdegBZ1c/g7P+DwrZtpHXK7ajZbcwpTEfFker8T+BrwsnYUdTzqr3sAupnZHFVX9NP/k+fTvWE9ja1Pc+BrdzHy8A73UpnNwqzDlKR+SYvHHgNvAu5vV2GTHtc9U2ZmbaGK6H7hyfRfdiHVZX0M/a8fcfifHqR10MvQmM3EXHqmVgG3SboHuAP4+4j4h/aUNTmPmTIza6/q0j763vwiul9+Oo3tezjwtbsY3rSFGPHyXWbTUZvtGyPiEeDFbaxlWio+zWdm1naqiO5z11Jft4KhOx5h+PuPM/zAVrrOXUv3uWtQdz3rEs1ya9ZhKivq76K53etMmdncSDoZ+GuSXvYAro6IP822quxVlvTS94YX0Nx9gOFNWxjZtIWRB56k6/lr6HrBWio9DlVmxyrePFOLu2CkyaHr7s26FDMrtgbwoYg4F3gFyRXJ52ZcU25UT1hE38Xn0n/ZhdTWLmfk3ic48OU7Gdr4KK0hj1s1G69wYWrRv38lAKMPzn3OKjNbuCJiW0TcnT7eD2wG1mZbVf5UV/TT96+fT//bL6R2ygpG7tvKgS/fyeH/9SMa2/f66j8zCniar7pqEZWTFtHaeTDrUsysJCStBy4Abj9m+5XAlQCnnHLKvNeVJ9Xl/fS97hyaLz6FkQeeZPTRQUYf3kFlSS/1s1ZRP+NEKv3dWZdplonChSmAyon9tAYdpsxs7iQtAr7JBlH4AAANoElEQVQCfDAi9o1/LSKuBq4G2LBhg7tggOqyPnpffRY9Lzud0cd2MfrQdobveozhux+jtnY59bNOonbyClQt3IkPs1krZJiqDvS5Z8rM5kxSnSRIXRcRX826niJRvUrXWavoOmsVzb2HGX14B6MP76Bx62bUXaN28gnUTllBbc1yVK9mXa5ZRxUyTFVO7GfkzqeyLsPMCkySgM8BmyPiU1nXU2TVpb1UX7I+WfPvqWcY/fFORrfsYvThHVCtUFuzjNopJ1Bbt4JKX1fW5Zq1XTHD1Mo+n+Yzs7l6NfA+4D5Jm9JtH4uIb2ZYU6GpIurrVlBft4JotWhu30tjy9OMPrGbxhNPA1AdWEzt5BVU1yyjesJiVFHGVZvNXTHD1NIeYv8IEUHyx6WZ2cxExG2AG5AOUaVCbc1yamuW0/3y02k9c4jGlt2MPrGb4bsfh7sfh1qV2qolVFcvpXbSMionLHK4skIqZJjSkm5oBXFoFPW7y9jMLM8kUV3RT3VFP93nn0Lr8EjSa7V9L81te2hsfIZhgHoark5aSnVgMdUTFqF6IX9N2QJTyJ/SypLk8tvYNwwOU2ZmhVLp7aJy2gD10wYAnhuutj5zdN9lfVRXLqY6sIjqysVUlvf7SkHLnUKGKaVhqrVvmOrqxRlXY2Zmc/GccDU0QnPXAVqD+2nuOkBj69PJYHaAiqis6Ke6vJ/K8qS3q7K838vcWKYKGaae1TNlZmalUunporJuBaxbAUBEEAeGae5KwlVz134aW3YTD+048h711pNwlYasytJeqkt7vUCzzYtChiktTk7ttRymzMxKTxJa3ENlcc+R3quIIA6P0nrmIM1nDqb3hxj5wTZoto6+t7tGZWkflaW9VJb0JvdLe6ks6kE1z39l7VHMMOWeKTOzBU0S6uui0tdFbe3yI9ujFbT2D9Had4jW3sPp7RCNrU8Th0ef/TV661QW91JZ1J2GtV4qi7uToNXX7SsLbdoKGaaOnObb7zBlZmZHqaJkEtGlvXDys1+LkUYSrvYdTgLXgSFa+4do7NhHPDoI4xcMEqi3i0p/N1rUTaU/uWnsvq8b9dQduAwoapga6Aeg+cS+KfY0MzNLqKuWTLkw8NwLl6LZIg4OHw1ZB4eT5weGae06QGPLbmgeszyjQD1dR3rIjtz3jt3qVHqSe59SLLdihqnF3dTOPoGRO558zmsxNMq+37qFrteeSs/bzvaknmZmNiVVK2hJMq5qIhFBDI0mAevgMHF4hNahEeJQct86MEzs3EcMNyY+QL1KJQ1Y6klv3cl9pefYbTWHr4IpZJgC6HrFOoa+8SNitPmsRTQPfuZuDnzqe/Cp76FlPSz9k0uonbaM7teuz65YMzMrNEmotwt6u6iunHxKnmi2iMMjyeD4oeT+2OetvYeJHfuI4dFnn1ocr1pB3bUkXHXXjj7uqaOu9HlXDXVVUVf9yHO6qu5EyEBhw1TPO57PoWvvYfjmH9Nz6fMAGLn7Kfb+2reO7BN7htjzs38HQNdrTuGEb76HyuLuTOo1M7PyU7WCFvXAoh6m6luKCGK4QQyPJr1eY7fhRnI/cvS+tedQum8DYrIEluqqJkGrPha20qBVrx4NYPVa0hEx7rG6qjD22BOjzsicwpSkS4A/BarAZyPiE22pahp6fuJM1Ftj6B+TMNXaO8SeX/p7AJZf9w5ae4epnbaMvb/+jzQ272Lkti3suvhall39VrrOXz1fZZqZmU1IEuqpQ08dlk7vPREBjWYSrEaaxEgavkYaR+4ZaSbPRxrEaDM5LfnMoeS10cbkvWHjVZSErXoV1apHH6fPj26vQG3c9vHbalVUqxzdVq2WdsD+rMOUpCrw58Abga3AnZJujIgH21XccY/fXaPrNacwdOMPaf7mq9l92RcY3bSd5df/G/re/cIj+/VcchYAh7/8AE9ffgODF/w3ai9axdJPvIGu151Kpc/L0ZiZWTFISnuPZvfrOwljLWK0SYymwWu0kT5vwsjRx9Fowtjj0eRx69DI0e2N1rPm9JqWqpLQVa2k4WvcfXUshFXS1ytH9xt7XquiqpJgVj12v/Q+3U5F83bKcy49Uy8DHo6IRwAkfQG4DJiXMAWw6Ddexe6f+Fu2r/0UdFVZ8dV30fvWsyfct/enX8Cql6zm4F9t5PD197P70utAyZWBWtSVdG+2SzmDtxkAtecPcMJX3pV1GWY2C0kYS3uamHtnQrTSnrLG0cAVjVa6rZUEr8a4baMtotlMAt3YtmYLGi1aQ6Pp9iY0W7MLa8eqjgtYaWDrvej5VJf2zfmzjzeXMLUWeGLc863Ay4/dSdKVwJUAp5xyyhwO91w9bzqTFTdezvBNP6bvvS+i6+Xrjrt/7fQVLP2jN7H4o/+K4VsfZfS+nTS37Sf2j0Bjjv9gY6Y6l21WcNX1y7IuwcxyQhXB2JisDoiIJFCl4epIwGqOC2XjXk+2T3QfRx534krJjg9Aj4irgasBNmzY0Pak0fvWsyftjZpMZXkvve84l953nNvucszMzKxNJMHYWKwcXz82l+H6T/Ls+WXXpdvMzMzMFoy5hKk7gbMknSapC3g3cGN7yjIzMzMrhlmHqYhoAL8C/COwGfhSRDzQrsLMzDpN0iWSfijpYUkfyboeMyumOY2ZiohvAt9sUy1mZvMm6+ldzKw8PMWpmS1UR6Z3iYgRYGx6FzOzGXGYMrOFaqLpXdaO30HSlZI2Sto4ODg4r8WZWXE4TJmZTSIiro6IDRGxYWBgIOtyzCynHKbMbKHy9C5m1haKeZyxW9Ig8Pg0d18J7OpgOfOh6J/B9WerLPWfGhG569aRVAN+BFxMEqLuBP6vya5KnmH7Bfn/98tzfXmuDVzfXOW5vmNrm1b71fEZ0MebSYMqaWNEbOhkPZ1W9M/g+rPl+jsrIhqSxqZ3qQLXHG96l5kGwrx//jzXl+fawPXNVZ7rm21t8xqmzMzyxNO7mFk7eMyUmZmZ2RzkOUxdnXUBbVD0z+D6s+X6iy3vnz/P9eW5NnB9c5Xn+mZV27wOQDczMzMrmzz3TJmZmZnlnsOUmZmZ2RzkMkwVYSV3SddI2inp/nHbVki6WdJD6f3ydLsk/Zf089wr6cLsKj9S68mSbpX0oKQHJH0g3V6IzyCpR9Idku5J6/94uv00SbendX5RUle6vTt9/nD6+vos6x8jqSrp+5K+kT4vWv2PSbpP0iZJG9NthfgZ6pS8t18T/ZtlXM+029Ic1XeVpCfT7+EmSZdmVNuM2vEc1ZeX79+Mfo8cV0Tk6kYy38uPgdOBLuAe4Nys65qgztcCFwL3j9v2R8BH0scfAf4wfXwp8C1AwCuA23NQ/2rgwvTxYpLJC88tymdI61iUPq4Dt6d1fQl4d7r9r4BfSh//P8BfpY/fDXwx63+DtJZfB/4H8I30edHqfwxYecy2QvwMdej7kfv2a6J/s4zrmXZbmqP6rgJ+Iwffuxm14zmqLy/fvxn9HjneLY89U4VYyT0ivgs8fczmy4Br08fXAm8ft/2vI/EvwDJJq+en0olFxLaIuDt9vB/YTLLIayE+Q1rHgfRpPb0FcBFwQ7r92PrHPtcNwMWSNE/lTkjSOuAngc+mz0WB6j+OQvwMdUgh2q88mWFbOu8mqS8XZtGO56W+XJjF75FJ5TFMTbmSe46tioht6ePtwKr0ca4/U3rK6AKSVF6Yz5CeItsE7ARuJukR2BMRjXSX8TUeqT99fS9wwvxW/Bx/AnwYaKXPT6BY9UPS8Nwk6S5JV6bbCvMz1AFF+IwT/ZvlzWQ/Q3nyK+np6muyPA05ZprteGaOqQ9y8v2b4e+RSeUxTJVCJP2DuZ93QtIi4CvAByNi3/jX8v4ZIqIZEeeTLFD7MuCcjEuaNklvAXZGxF1Z1zJHr4mIC4E3A78s6bXjX8z7z9ACddx/s7zJ6c/QXwJnAOcD24BPZllM3tvxCerLzfevXb9H8himiryS+46x0xbp/c50ey4/k6Q6yQ/4dRHx1XRzoT4DQETsAW4FXkly6mhsmaTxNR6pP319KbB7nksd79XA2yQ9RnIq6CLgTylO/QBExJPp/U7gaySNUeF+htoo959xkn+zvJnsZygXImJH+ku4BXyGDL+HM2zHc1Ffnr5/Y6b5e2RSeQxTdwJnpaPpu0gG296YcU3TdSNwRfr4CuDr47b/THo10yuAveO6YDORjrf5HLA5Ij417qVCfAZJA5KWpY97gTeSnI+/FXhnutux9Y99rncCt6R/sWUiIj4aEesiYj3Jz/gtEfEeClI/gKR+SYvHHgNvAu6nID9DHZLr9us4/2Z5M9nPUC4cM9bvp8joeziLdnxeTVZfjr5/M/09Mrl2jYpv543kqp8fkZy7/K2s65mkxutJuidHSc6pvp9kDMu3gYeAfwJWxNErBv48/Tz3ARtyUP9rSLp+7wU2pbdLi/IZgBcB30/rvx/47XT76cAdwMPAl4HudHtP+vzh9PXTs/43GPdZXs/Rq/kKU39a6z3p7YGx/6tF+Rnq4Pclt+3XZP9mGdc07bY0R/X9TfozfC9JcFmdUW0zasdzVF9evn8z+j1yvJuXkzEzMzObgzye5jMzMzMrDIcpMzMzszlwmDIzMzObA4cpMzMzszlwmDIzMzObA4cpMzMzszlwmDIzMzObg/8fH3AhkjAJVCAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "\n",
    "# 定义网络结构\n",
    "model = Linear(2)\n",
    "# 定义优化器\n",
    "opt = Momentum(init_lr=0.01, model=model, rho=0.9)\n",
    "train_and_plot(opt, 'opti-loss4.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.3.3.2 Adam算法\n",
    "\n",
    "Adam算法（Adaptive Moment Estimation Algorithm，自适应矩估计算法）可以看作动量法和RMSprop算法的结合，不但使用动量作为参数更新方向，而且可以自适应调整学习率。\n",
    "\n",
    "Adam算法一方面计算梯度平方$\\mathbf g_t^2$的加权移动平均（和RMSprop算法类似），另一方面计算梯度$\\mathbf g_t$的加权移动平均（和动量法类似）。\n",
    "\n",
    "$$\n",
    "M_t = \\beta_1 M_{t-1} + (1 - \\beta_1)\\mathbf g_t,  \\\\\n",
    "G_t = \\beta_2 G_{t-1} + (1 - \\beta_2)\\mathbf g_t \\odot \\mathbf g_t,\n",
    "$$\n",
    "其中$\\beta_1$和$\\beta_2$分别为两个移动平均的衰减率，通常取值为$\\beta_1 = 0.9, \\beta_2 = 0.99$。我们可以把$M_t$和$G_t$分别看作梯度的均值(一阶矩)和未减去均值的方差(二阶矩)。\n",
    "\n",
    "假设$M_0 = 0, G_0 = 0$，那么在迭代初期$M_t$和$G_t$的值会比真实的均值和方差要小。特别是当$\\beta_1$和$\\beta_2$都接近于1时，偏差会很大。因此，需要对偏差进行修正。\n",
    "$$\n",
    "\\hat M_t = \\frac{M_t}{1 - \\beta^t_1},  \\\\\n",
    "\\hat G_t = \\frac{G_t}{1 - \\beta^t_2}。\n",
    "$$\n",
    "\n",
    "Adam算法的参数更新差值为\n",
    "$$\n",
    "\\Delta \\theta_t = - \\frac{\\alpha}{\\sqrt{\\hat G_t + \\epsilon}}\\hat M_t,\n",
    "$$\n",
    "其中学习率$\\alpha$通常设为0.001，并且也可以进行衰减，比如$a_t = \\frac{a_0}{\\sqrt{t}}$。\n",
    "\n",
    "**构建优化器**  定义Adam类，继承Optimizer类。定义step函数调用adam函数更新参数。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Adam(Optimizer):\n",
    "    def __init__(self, init_lr, model, beta1, beta2, epsilon):\n",
    "        \"\"\"\n",
    "        Adam优化器初始化\n",
    "        输入：\n",
    "            - init_lr：初始学习率\n",
    "            - model：模型，model.params存储模型参数值\n",
    "            - beta1, beta2：移动平均的衰减率\n",
    "            - epsilon：保持数值稳定性而设置的常数\n",
    "        \"\"\"\n",
    "        super(Adam, self).__init__(init_lr=init_lr, model=model)\n",
    "        self.beta1 = beta1\n",
    "        self.beta2 = beta2\n",
    "        self.epsilon = epsilon\n",
    "        self.M, self.G = {}, {}\n",
    "        for key in self.model.params.keys():\n",
    "            self.M[key] = 0\n",
    "            self.G[key] = 0\n",
    "        self.t = 1\n",
    "\n",
    "    def adam(self, x, gradient_x, G, M, t, init_lr):\n",
    "        \"\"\"\n",
    "        adam算法更新参数\n",
    "        输入：\n",
    "            - x：参数\n",
    "            - G：梯度平方的加权移动平均\n",
    "            - M：梯度的加权移动平均\n",
    "            - t：迭代次数\n",
    "            - init_lr：初始学习率\n",
    "        \"\"\"\n",
    "        M = self.beta1 * M + (1 - self.beta1) * gradient_x\n",
    "        G = self.beta2 * G + (1 - self.beta2) * gradient_x ** 2\n",
    "        M_hat = M / (1 - self.beta1 ** t)\n",
    "        G_hat = G / (1 - self.beta2 ** t)\n",
    "        t += 1\n",
    "        x -= init_lr / paddle.sqrt(G_hat + self.epsilon) * M_hat\n",
    "        return x, G, M, t\n",
    "\n",
    "    def step(self):\n",
    "        \"\"\"参数更新\"\"\"\n",
    "        for key in self.model.params.keys():\n",
    "            self.model.params[key], self.G[key], self.M[key], self.t = self.adam(self.model.params[key], \n",
    "                                                                                 self.model.grads[key],\n",
    "                                                                                 self.G[key], \n",
    "                                                                                 self.M[key],\n",
    "                                                                                 self.t, \n",
    "                                                                                 self.init_lr)\n",
    "\n",
    "\n",
    "                                                                                 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**2D可视化实验** 使用被优化函数展示Adam算法的参数更新轨迹。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x1 initiate: [3.], x2 initiate: [4.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAF5CAYAAACFu8BrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XmcbHdd5//X9yx1aq/qrl7umpuQFSYIxMiAKIqKMMiPYXMGx3EY1AkwijjCjxH8aUSHAeenDIPDFhVkXIjI7jJoYBJCBAJJWJJAlpvt7rfX2uvs3/njnDq3um/33e+t7r6fp4+v3+Wcqv5Wk9v1rm+d+pbSWiOEEEIIsRkZ456AEEIIIcSZkiAjhBBCiE1LgowQQgghNi0JMkIIIYTYtCTICCGEEGLTkiAjhBBCiE3LGvcEAJRSjwMdIAJCrfX1452REEIIIc6n9Z77lVJvAH4pHf87rfVbTnQ/GyLIpJ6ntV4Y9ySEEEIIccGseO5XSj0P+JfA07TWnlJq5mR3IG8tCSGEEGKjeD3wLq21B6C1njvZDTZKkNHAPyql7lZK3TDuyQghhBDivFvruf8q4IeVUncqpb6klPqBk93JRnlr6Ye01gfTJaRblFIPaK1vHz0hfZA3AJRKpe+/5pprxjFPIYQQ4oK7++67F7TW0xfq5ymlzsX3F90PuCP9m7TWN430j3vuJ8klk8CzgB8APq6UepI+wfcpbYggo7U+mNZzSqlPA88Ebl91zk3ATQDXX3+9vuuuuy74PIUQQohxUEo9Me45nAH3RB/eWee5/wDwqTS4fF0pFQNTwPx69zP2t5aUUiWlVGXYBn4SuG+8sxJCCCHE+XKC5/7PAM9Lx68CcsAJPwi0EVZkZoFPK6Ugmc9faq0/P94pbT5xHBNFEVEUEYbhmu21yvB2a9WnUrTWx/WHY6P1egU4rj1an470v6EV9er2WsUwjBX1sD0sq/snKqZprluvVyzLWrNtGGN/nSGEEOfLms/9Sqkc8GGl1H2AD7z6RG8rAagzecIYt3P03p0QQgixoaz3nKyUuvtC7rF2jp5nL8icN8KKjACe+9znYlkWlmVh2zamaWb1cHxYRl+5r67Xe6V/opWB0ZWDtVYXVq9ODMfXW9FYa4XjRAWOXz05U+ut8pysrF5dWj0+bEdRdNwq1HqrWKMrXOuthJ1o5Ww4tlY9WqIoIgiC48Zvv/32E/2qhBBiS9iyQea5z30ujuNkJZfLrWgP+8P2sG/b9pr9YW1ZVnY727ZXHB+2TdMc98MX4oxEUYTv+wRBkNXD4vs+nucRhiG+72dleGy073neirHRvud5WX/Y9jxPgpcQ4oxsyiDzjGc8gzvuuGPc0xBiyzFNk0KhQKFQGPdUjhPHcRZ6XNfN6tXtE5XBYHBcezAY8IUvfGHcD08IcYY2ZZARQlx8DMMYS8iKoojBYEC/38+CT7/fX1H3ej1c16Xf79Pr9bJjq8stt9xyQecuxMVAgowQQpyAaZqUy2XK5fJ5+xlhGGYhaFgPy+p+r9ej2+2u6LfbbW677bbzNj8hNjIJMkIIMWaWZVGtVqlWq+f0fn3fp9vt0u126XQ69Ho9Op1O1h+tu90u7XY7G7v11lvP6VyEOF8kyAghxBaVy+WYnJxkcnLyrO8rjuMsCLXb7RV1q9Wi0+msaLfbbVqtFl/84hfPwSMRYn0SZIQQQpyUYRhUKhUqlQo7duw44/uJooh2u50FnWG72WxmY8PxYVvCkDgRCTJCCCEuGNM0mZiYYGJi4ozvw3XdLOQ0m80V9fLyctYfFnmbbGuTICOEEGJTyefz5PN5ZmdnT/u2cRxnb4EtLS1lYWd5eTkrw/7S0hJf+tKXzsMjEOeSBBkhhBAXDcMwqNVq1Go1LrnkktO6re/7WdhZWlpieXmZxcXFFf2lpSU+/elPn/H84jiW71k7TRJkhBBCiFOQy+WYnZ097ZWgfr/P0tISi4uLLC0tZWVxcTErS0tL3HLLLRJizoAEGSGEEOI8KhaLFItFdu3aNe6pbEkS/YQQQgixaUmQEUIIIcSmJUFGCCGEEJuWBBkhhBBCbFoSZIQQQgixacmnljYhrTVxHBNFEWEYrihRFK0YH7aH43Ecr+iPjq9Xr27HcZzNYXXRWmfHRuvRAmTjw8cz2h6tT4dS6rh6tD38WONwfFgMw1hRD9ury+px0zSPa69VjxbLslaMW5aV1cP2aH+0yMcyhRDieBd9kInjGM/zsuL7Pr7vZ/0gCLJ+EAQrzhkeW6sfBEHWH9ZhGK4YG5bh+Gh7tB4twzFxcbIsC9u2jws5w7HRenXbtm1yuVw2PmwP62F7dHytvuM42LaN4zgrznEcZ8WxYZEAJoQ4nzZlkJmfn+fd7343nufhui6DwSBru66btYdhZHV7GExc1z0voWD4x3z1E8RaTyxOACXTwSqVsS0rKaaNZZlYpoVt2VimiWWZ2KaNmY5bppkWK6sN01jRNw0jeZVvGBiGiZkeNw0D00hXCbK2gVJGNmYMx4crEMpIz1GYhokyFIY6dkypZMdMxcgKx6qVD8VwtWNlH45fPTlT663yaIYrRXpFP1tBGllJivVwFQliHR8bizVRHKG1Jori7FgUx0RxdKw9upIVR9lYFEeEaR1FMXF6LAzDbDyMIsIoJI5iwijM+kmdtKMwIogCwjAZC8IgOScdD8KQIAwJw4AwiPBVyGAwOC4srw7cw3KuWZZFPp9fEW6GZXR82B5uPz9sFwqFFePDMhwvFApZf3Xbtu1z/niEEBuLOpMl/HFTSmWTtm173T+A+XyefAA5O0fecXDsHE7OwcnlknY2lvRzdo6cbR/rj4w7dvqq1LbTsTSgWEk7Z9vYloVlWmf1RCzEOGmtCaMkCPlBkJQwXVUMArzAx0/Djxf4eL6PH6armGnfC9JQFAZZ3/O8Y8f9pO16Hn7g49oc9yJktB0EwRk/HtM0KRaLWbgZtkfr4fhoe3TsRKVUKlEoFGTVSZwzxWJxzXGl1N1a6+sv1DxGn2fPwgWZ86ZckXnqlU/m1j/6OI6dwzTNcU9HiC1DKYVt2diWTTFfGPd0AIiiKAs+A8/F9Vxc3x9pD8eTeuAm4wPPo+8NsvbAHSTH03Yn6DA3N8dgMKDf72f1maxKDUNNqVSiWCxSLpezejheKpWO61cqlRV1uVzOiuM45+G3KcTWsymDjGWaG+aPrBDi/DJNk6JZuGD/5qMoou8O6LsuA2+QtfuDPj13kNW9QZ/+YLBirD/o0x3003rA/Pw8/X6fXq+XlVNl2/aKYDMslUrluHrYrlarx9XDkCSrRmKr2pRBRgghzhfTNKmUylRK5XN+33Ec03cHdAd9ev1eWvfpDnr0Bn06/R7dXo/uoEe336Pb79Pt95LxfpdOr8eRI0fodrt0u106nc4pXeenlMoCT7VazQLOaL26Xa1WqdVqK9r5fF7eOhcbjgQZIYS4QAzDoFwsUS6WoDF91ventcbzfdq9Lt1+N617a9adXjetO7S7XVqtFvv376fdbtNut09ptci27ePCzbBfq9Wo1+vZ2FrtSqUiK0PinJMgI4QQm5RSirzjkHccZiYbZ3VfYRjS6fdodzu0e13avbTudmh1O0k/bQ9L2/d4+OGHabVatNttut3uSec7DDfDgDMstVqNiYmJFWOTk5PU6/Vs3LLkKUscT/6rEEIIgWVZTFRrTFRrZ3wfYRjS7nVpdds0O+2R0NOm2W7R7HZoddo0O63keHgsCDWbTfr9/gnvv1qtMjExsWaZnJxcUQ9Lo9Egl8ud8WMSG58EGSGEEOeEZVlM1upM1upndHs/8FluD4NOi6V2KwlAnTZL7SbNTpvldpOldovlfp9Dhw6xuLhIs9k84bVC5XKZycnJLOQ0Go2sPyyNRiMbbzQaVCoVuR5ok5AgI4QQYkPI2TlmG1PMNqZO63Zaazr9LsutFkvtJsvtFsvtJoutJkutJkvt5bRusdTrsX//fpaWllheXl7361Bs214RcNYrU1NTWV0sFiX8jIEEGSGEEJuaUopqqUK1VGHPjl2nfLsoimh22yw2k6Cz2EoDT2uZxdYyC8PxsMcDDzzA4uIii4uLxHG85v3l8/ks2AzDzYnKxMSEXPx8DkiQEUIIcVEyTZNGbYJGbeKUbxPHMa1uh8XmMoutJeaXl1hsLbPYXGahucRCWi92Ojz22GMsLCzQ6XTW//mNBtPT01m5+eabJdycJgkyQgghxCkyDCO7KPoKLj2l27iex0JriYXlpZHAs8Tc0mJaL7Aw6PLAAw9IiDkDEmSEEEKI8yjvOOya2c6ume3jnsqWJNFPCCGEEJuWBBkhhBBCbFoSZIQQQgixaUmQEUIIIcSmtWGCjFLKVEp9Uyn1t+OeixBCbCQ3f/5zXP3S51J69pVc/dLncvPnPzfuKQlxTqx+7ldK/bhS6h6l1LeUUncopa442X1smCADvBH43rgnIYQQG8nNn/8cv/Sut7H/yCG01uw/cohfetfbJMyIrWL1c/8HgJ/VWj8d+Evg/zvZHWyIj18rpXYBPwW8A/i1MU9nU9FaE0URYRQRRuFxdRSmdRxn48Pz4zjK+lEcE430Yx0TRTE6rZN+RKw1URwRx3FStCZOb6tjjSbpa62JtU7qOB4ZT+Y87A8fg0Znx4Zjp2u4NbhSCqVAobIxwzCyvlLH+oZhoJTCUMmx7DxDYRomhmFgpOOGYSRjSmGaJoYyME0DldaGMrBMC9NMzjMNA9M0sUwTw0jqYd8yLUwjPd9K+sPxYW1bVjY/cfG68YO/z8B1V4wNXJcbP/j7vOqFLxnTrIQ4e+s892ugmrZrwKGT3c+GCDLAe4C3AJX1TlBK3QDcALB7247zPqEoivACH8/3k9rzsr7re/iBjx8E2Zg/Wgc+QRDgh0E6FhCEAX6QljA9np4ThCFhejwIQ4IwxA99wrSdlOF5IWGUjIVRRBAG5/13cSEppVYEklM1GoDOJARtZLZlY5kmtmUlQcdKQo5t2WmdjOWsXNbP2TZWejxn2eRsG9u2yVk5cnbaT8edXC45Z1jbuWQsrR07KbZtk885x8YcJ6lzOfI5RzbyOk8OHD18WuNCbCBTSqm7Rvo3aa1vGumv9dz/i8DfK6UGQBt41sl+yNiDjFLqxcCc1vpupdSPrnde+uBvAnjyk67UX7/vmww8l4Hn4boufW/AwPPwPC8dd3HTtut7eL6XjXm+v2I8qf20nQSScxkQTNPEsdMnECt9Qhk+kVh28iSU9q2WR8G0sM0ilm1iOxa2aWIZJraRvFq3jaRvmSaWMrFNE3M4lhUjGzOVkfWTdrLSkI0rY6SfrCQopTBVekwZ2ZgxHDeScaUUBgpDpSsXw5UNpUZWQFb1Ib1tsmqS/ndwzn7fQ1m4SVd7Yh2jGVkRSkPP6n6k4+z8mHRVScfJqpWOj50XpytVaRmOhXGUnJeeHw5vN3IsiqNsPIyjrB/GUVaiOCKIIkKdrrjFEUG6ahbEYdJP20EUErgRYeQRTRi0up0s+CZhOQ3KaXgehvAois7Z79u27DTU5HDSwJMEH4d8zqHg5JMxZ9hO6nzOycYKTh7HcSim7Xw+T8FxKDgFCk6eYj5P3slTzBco5vPk7NyWXrH6/FduQ6HQHB/Od83K5mri/HnGM57BHXfccVb3USqVFrTW16917ATP/f8JeJHW+k6l1P8LvJsk3Kxr7EEGeA7wEqXUi4A8UFVK/bnW+t+ud4MHHtvLj/7iT5/0jod/SPPDP4rpH9iCk8du+zQsG8cuki/kcCwbx7TJW8faOcvCMXM4lkXOtHEsm5xppcdWth3Twh72zeRYzrKwjeQtBHHhZSs7KFBgbqhLwjaGKI4J4hA/DPGjED8K8KKAIArxoiQEeVGQHBtpe2GQnBuGeJGPH4Z4UYAXBrihv7IdBHgDj35Vs9hazl4wJC8skhcdfnBmLxwMw8gCTsFJwk2xUBzpHxsr5QsU8gVKhQKlfJFCPk+5UKJYSMaK+WJ2rFQsUi6UcHLjCUp+4PNb7/993vuxD7NrZjsLzSVc38uOF/J53v66N1/weQlxDq313P93wDVa6zvTc/4K+PzJ7mjsQUZr/VbgrQBpKnvziUIMwJ76LO/8yddQsHIUbIe8nSNv5ShYSZ30bQwlT1xCnIhpGJhG8u9mnGId44YBbuDjhj6DMKndwGcQeFl/kB7vBx6DwMvG+oHLIG0PfI9+r8dS2ePA0UP0XZeBO6DnDui7g9N669EwDMqFIsVCkXKhSKlQpFwspXWRUqGUjBeLVIolyoUS5WKJcjE5r1IsUyoUqZRKab+Ek3NO+DP37nucV//Wr/LNB+7jta/8t7zzDW/lM7f+Azd+8Pc5cPQwu2a38/bXvVmujxGb2lrP/cBLgSNKqau01g8Bz+cUPgQ09iBzJur5Es+/4rpxT0MIcY4YyqBoOxTtEz/Jny2tNW4Y0A9c+oFH3/eydi/w6PvpeODR8136gUvPd+mldT/wcE2TxdYy+w4foDvo0xv06fR6hFF4SnOwLZtKsUSlVKZaKicBJ23PLS3w1e/cg2kavPr/+Wmecc21fP6fbmVmssGfv+MPqZUrVEplauUKWust/baauPhorUOl1H8APqmUioFl4OdPdju1GS+MfPr2y/UXfv5d456GEEIASUDyo5CuP6Dnu3R9l64/WNn3BulYcqzj9dN6QNuJeezgfjq9LoZSxKfwd9kyrSzYVMtl6uUq1XKFWrlCLW3Xy5W0rlKrHBurVarUyhVsy74Avx1xOgr//PI1x5VSd693vcn5cN111+lzcI3MBZnzplyREUKIjUQplVxbZ9k0itWT32DEt488yg2f+R/0+j3e/EOv5E0/9AoAOl4Sdjr+gLbbz9odr097WNwkDLXcHr045olDB2h1O7S6bdq97knfRisVitTKFSaqNWrl6op2vVplolKjXqlRq1Sydr1aZbJap+DkZUVIbAgSZIQQ4gL6xH1f5h23fYyD7UWq+SIdt89sZYJP/Zsbec6ep2TnTRTKTBTKZ/xzYh3T9VzaXp+W16Pl9mi7x9odr0/TPTbetQwOzR/lu48+TLPTotXtnPD+c7ZNvVJjolJjojpa6ll7MmvXmUzrWrmCaZpn/LiEWE2CjBBCXCCfuO/L/Nrff4hB6APQcnsYSvGm57x8RYg5FwxlUM0XqeaL7GLqtG8fxfGKsNN0u8e1m4O0bnc55B3l/kceotlp0e51171fpVQWfiZrdSaqdRq1OpO1CSZrdSardRrDdq3OVH2CRm2SQj5/Nr8OsYVJkBFCiAvkHbd9LAsxQ7HWvOcrn+HV1/3kmGa1NtMwqBfK1M9gVSiIQlpun6bbZXnQZdnt0hwk7abbZWnQYXnQYbnfZT5a5MHHH2GptUyn31v3Pov5Ao36BI3aBFP1CabqkzTqkzRqEzTqE0zXJ5mamKSRHpus1mXl5yIhQUYIIS6AMI440F5Y89jB9uIFns35ZZsWU6UqU6XTu17Ij8Ik+Aw6LA06LPU7abvLYr+djA06NLsdHj24j4XlpXVXf5RSTFbrTNUnmZ6cZKo+yVS9wdTEJNP1SaYnGul4g+mJBo2aBJ/NSoKMEEKcZ/tb87z2s+9d9/jOauMCzmbjypkWs+U6s+X6Kd/Gj0KWBh0W+20We20Wh+20LPTbNIl54LG9LDS/zmKrueZF0EopGrV6EnAmGsxMTmXt6ckGM5MNZtLxmckpSoXiuXzo4ixIkBFCiPPobx+4k1/9+w8SxTE/f90L+Nh3bl3x9lLByvEbP/ozY5zh5pYzLbaVJ9hWnjil86M4zoLPQq/FQhp2FnotFvtt5nstlqOIbz/0XeaXF2l22mveTzFfSEPNsXAz7G+bnM76s40pysWSfMLrPJIgI4QQ58Eg8PmtL3yUP/3mLTxj++V86KVv5LKJbfzArquyTy3trDb4jR/9GV557Q+Pe7oXDdMwmC7VmC7VYHr3Sc/3wiALOPP9FvO9Fgu9pJ7vNZnvtnh8sJ+v3/tN5ptLa672DEPPbGOK2TTkzDam2DY1w7bGdFbPTDYonI8HvcVJkBFCiHPsgfn93PCZ9/C9+f388rNewlt/5FXkzOTP7Suv/WEJLpuIY9nsqDbYcQpv/0VxzGK/zVyvyXyvldXzvSZz3SaLTswjBx7nn771DRZby8fd3jItXM+Va3VOkwQZIYQ4R7TW/Pm3vshv3PKnlHMF/upVb+PHnvT0cU9LXCCmYTBTrjNzCtf4+FHIfK/J0W6To91l5rpN2l5fQswZkCAjhBBnaHRzu+2VSbaV69xz+BF+5NKn8r6XvOG0LloVF5ecabGzOsXO6unv8SNWkiAjhBBnYPXmdoc6ixzqLPLSJz+bD730jRjKGPMMhbg4yL80IYQ4A2ttbgdw18GHJcQIcQHJvzYhhDgDBy+Sze2E2OgkyAghxGn64iPfWndfENncTogLS66REUKIU+RHIf/1to/xvjv/hh2VSRYHHbwwyI7L5nZCXHiyIiOEEKfgseUjvPh//Sbvu/NveM11P8nXXvde3vOi17GrOoVCsas6xbtf9FrZI0aIC0xWZIQQ4iQ+df8dvOl//xGmYfCRl7+JF1/zzwHZ3E6IjUCCzBaitSbWmiAOCeOYMErrYT+OiOIoaeso7cdZHcURkY4JR9pRHBPpmHhYa53WyVic/szhmNagOTYep7cZzg101tfpnDU6vV3yf8ljGXlcHL/l95Di2HUKw0sW1PD/VNpWyVlKKQylgKQe9g1lpLVCYaAUGMrAHI4bRnbecMxUBoaR9M2sNrFG2qZhYI3UlkqPG2bSNwwsw0pqM6ltw8rmJsav57u87ZaP8JffvpUf2HU1H/qXv8Lu2vS4pyWEGCFB5jTFOsYLQ7zQx40C/DDACwPcKMAL/eRY5OOHIW7o40chXnresB1kYyF+FGb9IEr7cYgXhoTxseNBHGXHw7QdxtGqfnzCJ/2NaBhEVoeOZEwdd96o0cc6/H6T1eFo9XmbgUIloca0kpIGn9H+aJ1Lx3OmhW0c6zumnY3nrGN9x7TTMRvHtHHSOjknh2NZOFaOfHosZ9lpOzm2lT9aPLrB3UypBsBcr8V/+sGX85bn/jSWIbuuCrHRbIkg40chg8Cjn5ZB4DEI/KQOPfpp2w193HA47jMIfNwgHUuPDftuGOCGPl56zEv7fhSekzlbhknOtHAse80nH9s0yZk2uQhKhkPOspIns5FiGSa54ZPcaK1GVwCGr/yPjZvKyFYOsr4yMNJxS5kYRrLqYKbj5nAVYtXKxHB1QimFwerVjVV1es5oaLlQRsNNzLEVIs3Keri6pNNzhqtQK1ai0naUrjhFerhqpdOVruS8MB0PR1e60n44rONhP1khC0b6QXyshPHxfT8OCaKIUMX0gx5BFB0Xiv0oSOsk8J4LWfixbApWLgk9Vo68nYSfvJ30C+l4wU6OFawcBdtJx5K6aOcoWA7FnEPBdpK2nRwv2g62eeH+RK3e4O5orwnArz77ZbztR191weYhhDg9mzLIfG9+H9e+97VZeDmTP9CWYR77Y5v+4c3+GMeKul0iX5jAMS0KZg7HtMhndfpKNg0e+bSdy8YscsZIe6SfMyxyprmlX9VuRCoNVCi4WF9TxzrGj5IA5EUhfjxcJTzWz9pR0najAC8ajqcrj1GIG/l4Ucggrd10hdIloOn2shcNbnDsRcKZ/jst2k4WbIq2QzGXpzTat/OUciNlnX45l6ecK1B2CtkXOI5ab4O7T9z/ZX7jefJJJCE2qk0ZZIpmjp/Y/hSKaRAZ1iva5rGxvGlnx/KmTd7KYcsSsbjIGMogbxnkscfy84M4ygJPf7gKGgXJ6mjoM4iSenjsWO1l7V7a7sQDjnaXk1VY36MXuPQD75TnYhsmZaeQBJs04BxYZ4O7A+0FPnn/HVTSEFRxilRyBSppe61QJIS4cDblv8BLKlP8wXN+dtzTEEKcBtswsXMFKhTOy/3HOmYQBvRDj17g0Qs9uoFLL/DoDscCNxvvBmntD/ju8qET3vfrPvvedY/lLZtKrkjZKVB1ilmp5I+1a/kilWHbKVHNF6k6JWrpORfyLTQhthr51yOE2BIMZVCyHUq2w/QpZqUj/Sb/8faP8ER3geunL+O+pQO40cgGd6bNb17/Mp6748l0ggG9wKPtD+gELp1gQMcfrd3kWBSwsHyYtten7fXpeIOTzqNoO2ngKVHNl6g5RepZu0S9UKLqlKjnk1IrlLN2OVeQT7mJi5oEGSHERemW/ffyxjv+jEHk857n/Bz/+opn8alHv8E77/kcB3tL7CxN8tbrXsIrLn/mWf2cKI7pBi7tYEDHH9DyB3SCpG6npeX303pA2+8z32uxd/EQLa9Hy+2lWxeszVRGEmoKZer5clqXkna+xEShwkQhGZ/Il5N2vky9UJJPYYktQYKMEOKi4kch77jns3zo/i/ylImdfOhHfoEr69sAeMXlzzzr4LKaaRjUnCI1p3hGt491TC/waPp9Wl4/qVe1l72k3/L7LPbbPLJ0iOagS8vtn3D7gapTZDINOpPFKpOFMpOFyki7SqNYWTEmb4OJjUb+ixRCXDQea8/x2i99mO8s7uM11/wIN17/cvLWeC5+PlWGMpKLi3MFdpdP7wspYx3T8gc0vR7LXp+m16Pp91nyujS9PsteLyluj4Vei4cXDrA46NDz3XXvs+oUaRSTgNMoVpksVJgqVtOxpIz2S7n82f4KhDghCTJCiC3pk498fcXbRD+x61r++pE7sQ2TDz/vBl605+njnuJ5ZyiDCafEhFPistO4nRcFLHs9ltwuS16PJa/Hotth2eux6HZZcrssul0Othe598jjLPRb6+6xVbSd48LNdKnGVLHKVKmWtpO6UazKp8DEaZP/YoQQW84nH/k6b/7KXzKIkn1hDvSW+NMHb+dJlRk+/oJfYVd5cswz3Ngc02Zbsc62Yv2Uztda0ws9Ft0ui26HhTToLLgdFt1O1p7rNfne/D4Wem28kYuqR9XzpSzYTJdqTJVqzJRqTJfq2diwXbSdc/mwxSYlQUYIseW8857PZSFmlBcHEmLOA6UUZTtP2c6zpzJ10vO11nQDlwW3k5RBEn4W3A7zgzbzboeFaMD35vdgS0EBAAAgAElEQVQz//i9NN3emvdTzhWYLtWYKdWZKdeZSQPOTLnObKnObHmC2XKdqVJNLmzewiTICCG2nIO9pTXHD/WWL/BMxFqUUtl1P5dVZ056vh+FacjpMO+2mRu0WRi0mR90knbs8sD8fm5//F5aa4QehWKqVGW2PMG28kRWb6tMsK08ybZKMjYtgWdTkiAjhNhSml6fvGkzWOOti50lWY3ZjHKmxY7SBDtKEyc91w2DJOz028y7bY72WxwdDOsWc70m3znyGAv91nEfa1copks1tlcm05AzwfbKJLNp2NlemWR7eZKJQln27tlAJMgIIbaMb8w9wuu+9BHcKMA2TIKR73cqmDneet1Lxjg7cSHkLZvd5cZJP+EVxhELbocj/VYWco70WxzpNznSb3GgtcBdBx5icdBZ82dsK08mwSYrDbZXJtlRnWRHpcFMuS6rOxeIBBkhxKYX65j/ee8/8nvf/Ft2lib4u596C4+158755nZi67AM85QuaPaigLlBmyP9Jof7LY70mhzuN5PAE/a45/Bejjy0hBuuXAE0lGKmVGdHNQk4O6tT7KgkIWd7tcHOaoNt5QnZl+cckN+gEGJTm+u3+OUvf5TbDz/ASy69jt//wZ+lmitw3fSlElzEWXPMk6/waK1Z9noc6Tc51G9yOA07h3rLHOk32bt4iC89di9df+XXVSgUM+U6O6sNdlSScPM/3/4yLEuemk+H/LaEEJvWbQe/yy9/+aP0Apc/+MGf5d9c+YNy7YK44JRSTObLTObLPGVy17rndfwBh9KAc7i3zMHeMof6yxzqLfPgwgG+cfAhPigh5rSN/TemlMoDtwMOyXw+obW+cbyzEkJsZEEc8d+++Tf84b3/yNX17XziBW/kmokd456WECdUyRW4Olfg6vr2cU9lw1BKmcBdwEGt9YuVUpcBNwMN4G7g57TWx++lMGLsQQbwgB/TWneVUjZwh1Lqf2utvzbuiQkhNo7RnXptw8SPI37uqh/i7c98JUUrN+7pCSHOzBuB7wHVtP97wH/XWt+slPog8AvAB050B8b5nd/J6UQ37dppWf9bzoQQF53hTr0HektowI8jcobJs2avkBAjxCallNoF/BTwx2lfAT8GfCI95aPAS092P2MPMpAsLSmlvgXMAbdore8c95yEEBvHf73ns8ft1OvHEe+853NjmpEQ4hx4D/AWIE77DaCptR5+cdcBYOfJ7mQjvLWE1joCnq6UqgOfVkpdq7W+b/QcpdQNwA0Au2RTKyEuGntbRzm4zo686+3gK4Q4S4MAde/hs72XKaXUXSP9m7TWNwEopV4MzGmt71ZK/ejZ/JANEWSGtNZNpdStwAuB+1Yduwm4CeBpU3vkrSchLgIf3/s1fv1rf4WBIl7jHWfZqVeIDW1Ba339OseeA7xEKfUiIE9yjcz/AOpKKStdldkFHDzZDxn7W0tKqel0JQalVAF4PvDAeGclhBinXuDyhi9/lF+543/xtMYl/O4zX0nBXHktjOzUK8TmpbV+q9Z6l9b6UuBVwP/RWv8scCvwyvS0VwOfPdl9bYQVme3AR9OPYBnAx7XWfzvmOQkhxuT+pQO89rY/4dHOHG962ov4tae9CNMwqDsl2alXiK3vPwM3K6X+C/BN4E9OdoOxBxmt9XeAZ4x7HkKI8dJa86cP3s5vf/2TTORL/PVPvpHnbL8qO/6Ky58pwUWILUhrfRtwW9p+FDitf+hjDzJCCNHy+vzaV/6cv3viW/zYzn/Ge3/43zGVr4x7WkKITUCCjBDighvd3G4qXyWMIzrBgN+6/mW87p/9OIYa++V7QohNQoLMJhDFMV4c4kchQRzipbUfRwRRiL+iHRHGEX4cEsYRwUgJR2udtKM4TmodE8YR4bCOY2IdZ/1hO9IxUZzWOibWOhuL0cRao7XOxmM0Oj1PA7HWxCN9rZPPomg0Ov1Qymh72F+P4tj36iiV9IdftaPSo0oZGCppGyvaCqUMTKUwSMeVwlAKA4VpGJjp+aZK2sMxSxmYhpm1DWVgGQaWYWKppDaVgW2YmEZS28rEMkxs41h9rFgr+6ZFzjDJGVbWtg0Lx7SwDYucaeEYFqax+Z7wh5vbDfeFmXfbKODNT/8p/uO1zx/v5IQQm44EmdMQ6xg3ChmE/rES+Sv6bhQwiIKs7YYBbuTjRSFu6ONGIV4U4EZBVvtREk78OMiO+1GYhZdIxyef3BmyR554h0++lmFgKjOtjZGx9Al75AndnqjgDPtGctwwFIYy0lCwcmw0LAyPQ/Kla0qpLJgoxYov/xsNLEOjAUfrJPwMx3QaqIb/uw3D1bCOdUwcD0PV8HgylgS1KDseLHcI4oAoTsJcnIa4cCTQhVkYTPpJSEyCYRBH5+1/P1MZWahxTJucaZE3LXKGjWNaad/GMe2R2iJv5XBMi7yZI2/ZFLI6Oa9g5Y4VM7einzets1oxeec9nztuczsNfOzhr/Kmp//UWf5GhBAXmy0ZZKI4phd6dAOXXuDRC72s7q/q9wKPfpgeC336oUc/8OmFHoPQp5+FFI9BFJzRfGzDJJ89yeQopE8ojmlRbNQp2Tkcy8axczhWLqltm5xlp/2kPSyOtbKfsyxsM6lzlo1lWulYMm5bZtq2sMzhuIlpmPJNwReA1poojgiiZNUsjEKCYQkjgihIVtPC5JgfBvhhSBAlddJPijesA58gCvGCAC/007Fk3Av9tA5wfY/2Ugsv6uBGSZB2Ix8/CnGj4IxD1jDgFNNwU7RyFC2Hop3UJcvJxkp20h/WB9bZxE42txNCnIlNGWQOdBd57W1/QjdMgko3cOkGw+DinlbgyBkWRStHyXbSP8pJPbtjO0WnQDHnUHQKFHIOpazOU8jlKdgOBcehkMtTzOXJ53JJbTsUcknJ2zksc1P+msU5opTCSkNkAWfc01khjEIGvo8beAx8Dzfw6Psuru/T910GvsvAS8cCj57nMvA9et6Age/R9wb007ozt8j8oMMgXEhfICQvBPw4PPlESFZlLvuzN1Ky85TtPGXbSeukVOw8JduhYheo2HnKuWPjFTtPJZeMV3MFSpYjIV2Ii8SmfIZt+wPuXdqf/bHbfcklVPIlSvkC5XyRSr5IOV+k5CT9slOgXChSzOUp5wuUnAKlfIFSroBtbcpfgRDnhGVaVAoWlULxvP2MIAx5dP4gb/jw73HnI/dx7a7LeejIPvzw2AsO27T4F09/Dnumt9N1+1lpHZlnbtDi0fYcHd+lG7oMQv8EPy1hKJWEHbtANZenaheo5pJSyRWyfi1XTOsC1VxxRZ2TFyBCbAqb8l/qtZdcyZdu/KNxT0MIcQpuf+Aebvij/0Lfc3n/z7+Vf/OcF/LXX/sCv/OpmziwOMeuxgy/9fIb+FfPPrULfcMopOsN6Az6dAY9Om6Pdr9HO62z/qBHe9ClM+jTGnQ5MrfAg83DdAKXtj846bVnBStHPVeklgabmpO067kidadEzSlkxyecEnWnRN1JjluGeS5+dUKIU7Apg4wQYuMLo5B3fObDvPvv/pyn7LyMj7zu7Vyz81IA/tWzn3/KwWU1y7SoFyvUi2e+z4zWmq47oD3o0uqPlEGXZq+T9js0+12a/Q5Lh45wsLvM/f5BWn6fbuCe8P6rdoG6kwScSafMZL7EhFNiYqTdcMpM5svZcce0z/jxCHExkyAjhDjnDiwd5Rc++Dt8be+9/Lvnvpjf+5lfoejkxz2tjFKKSqFIpVBk5+TMad8+jEJa/S7LvQ7NXoflXptmP6mXex2Wui2Wum2Wum0Wjhzh0c4cS26XzgkCUNnOM+mUsnDTyA9LJamdMlNpe6pQkeuAhEhJkBFCnLWPf/WW7K2iRqVG33MxlOKPb/hNfvpZW29vGMu0aFTqNCr107qdHwYs99osdlos9dosdposdltJv9tisdtiodNk/vARHmweZsntrPvhBcewmCpUsrAzlZbpwmi7mrQLFWx5u0tsURJkhBBn5eNfvYVf+eh/Y+B7ACx0miil+N2ffv2WDDFnI2fZzNYazNYap3ybnjdgodNkodNkMa3n2810bJmFTpMjBw/xUPMIC4M23jqfEptwSiuCzkyhykyhylRazxSqTOerEnrEpiNBRghxVn7nUzdlIWZIa82HvvhJ3vDCV41pVltHyUk+ablnavtJzx1e+zPfWWa+vcRcazltLzPXWmKuvczR/Qe4d3E/825n3Wt9Jp0ys8U03BSqzBZqzKT1bLHGtmKNmUKNkr2xthMQFycJMkKIs7J/8eia4wcW5y7wTMTotT9Pmtl50vP7nstce4n59jJHW0vMtZc42lxMAk9rkcP7D7C3Ncf8oL3mfkBVu5AFm9lije3FOtuK9RX1dKEin+IS55UEGSHEGfECn9/8+AfWPb6rcfoX0YoLq+jkuXR6B5dO7zjheVprlnsdjrYWOdpa4vDyPIebCxxpLnKoOc+hffv56pG9HO03CVd9rN1QitlCjW3FOjtKx0LOjtIE29N6W7Emn9oSZ0yCjBDitD06d5DXfOC3+dYTD/IT1z6Tf3ro2yveXirkHH7r5TeMcYbiXFJKMVmuMlmu8uSdl617XhzHLHSaHGrOc3h5gUPLC1noObg8x8MHD3L7oQfW/PTWVL6SBp0JdpSSsjOtdxQn2F6qy7U7Yk0SZIQQp+Uzd93GGz7yeyil+Itffgcvvu6HV3xq6XQ3uBNbh2EYzNQmmalN8vQ9V697XmfQ53BznoNL8xxanufg0hwHl+c4tDzPvv37+NrRh2n5gxW3UShmi1V2liazkLMra0+yqzzJpFOSj6RfhCTICCFOiRt4/MbN7+OPb/0M1z/pKXz4dTdmF6CezQZ34uKTXMezh6u271n3nK7b5+DSHAeW5lbU+x55lO8uHeCW/ffirvpoesHKsWsk2OwuN9hdbnBJWs8UqhJ0tiAJMkKIk3rk6H7+/Qd+m+/se5hffsG/5sZX3EDOkmsaxPlTzhe5eselXL3j0jWPa61Z6rbYv3SUA4tz7F88woHFo+xbPMq+Jx7n3n37WXS7K26TN212ldKAU0kCzp7KFJeUp7ik0qCeK0rQ2YQkyAghjjP6VtFkuUrX7VNw8tz8K+/kXzz9OeOenhAopbJNCdd7G6vvuexbOMK+xcM8MX+YJxaP8MT8YR5//FG+tfgEy15vxfkVO8+eylQWbo61G+wqT8oFyRuUBBkhxAqrN7hb7LYwlOLtL3mNhBixqRSdPNfsvDT7jq/VWv0uTyykIWfhcNZ+6InH+ML++1ZsLqhQ7ChNcGkabpJ6mksr01xamaLmnL9vkBcnJkFGCLHCWhvcxVrzvn/8K17//FeOaVZCnHu1Ypnvu+RKvu+SK487FscxR1tLPD5/iMfnD/FYWj+y92H+cf+9LLidFedPOKUs1FxaTQLOZZVpLqtOM5WvyFtW55EEGSHECrLBnRDJJ7C2T0yxfWKKZ1/1fccd77p9Hp8/zGNzB3ls/iCPzR1i74MPctf8Y3z28buJtc7OLdt5nlSd4fLqTFLXZrN+JVe4kA9rS5IgI4QAjn0qaT2ywZ0Qx5TzRa7dfTnX7r78uGN+GLBv4QiPzh3g0aMHeWTuAA9/7wHunn+Mzzx2N5pjIWemUOXy6iyX12a4vDrL2/xfIJfLXciHsulJkBFC8OjcQf79B27k2088xPOf+izuePCbssGdEGcoZ9lcsW03V2zbfdwxL/CT1Zuj+3n4yD72HtnPQw89yP/e923cMOBGWy4oPl0SZIS4yH32rtv45XSDu798wzv4qWfIBndCnC+OnVv3AuTOoC/X0pwBCTJCXKT8MOA3P/5+PviFT/L9lz2Zj7z+t2WDOyHGqFKQTz6dCQkyQlyEnlg4zL//wI3c89gDvP4nXsnv/KvXywZ3QohNSYKMEBeB0beKGpUaPXdAzrL5s1/6XV7y/T8y7ukJIcQZkyAjxBa3eoO7hU4TpRS/+fL/ICFGCLHpGeOegBDi/FprgzutNR/4wl+PaUZCCHHuSJARYos7IBvcCSG2MHlrSYgtKooj3vXZPx3Zemsl2eBOCLEVyIqMEFvQXGuJl/3Bm/lvf/NRnnPV0yjknBXHZYM7IcRWIUFGiC3mKw99mx/+7V/gzr338j9f85/5+1//Q9776rewuzGLQrG7Mct7X/0W2SdGCLElyFtLQmwRcRzz3n+4md/55B9x6fR2PvGf/n+eeskVgGxwJ4TYuiTICLFJje4Ns2Niikalznf2PcxLr38ef/iat1AtlMY9RSGEOO8kyAixCa3eG+bg8jwHl+d51bNfwAd/8W3yfS1CiIvG2K+RUUrtVkrdqpT6rlLqfqXUG8c9JyE2urX2hgH4p4e+JSFGCLHhKaXySqmvK6W+nT73vz0d/wul1INKqfuUUh9WSp30u1PGHmSAEHiT1vopwLOAX1JKPWXMcxJiQ1tvDxjZG0YIsUl4wI9prZ8GPB14oVLqWcBfANcATwUKwC+e7I7G/taS1vowcDhtd5RS3wN2At8d68TEceI4JohCgigkjCOiKMraQRgRxcMSE8UxYRwRj/RjPaw1URyhtSbWMVqT1ppYa7TW6HT3E63Teo35DNcdhisQCoVhKBQKpRSGMlCKtFaYhomhFKZhYCgD00iKYZhYhpn1TcPENExsKxm3TQvTPNa2TQvDGN9rgIcP78M0DcIoOu6Y7A0jhNgMdPLHvZt27bRorfXfD89RSn0d2HWy+xp7kBmllLoUeAZw5xrHbgBuANjdmL2g8xq3KI7oex4D36Xvu7i+zyDwcH0v6QceA9/HC3zcwMcLPAbBsO/hBT5eGCR1EOCFPn4Y4AUBfpiWKMQPfPwoJAgD/DDEjwLCMCRIA0us43H/KjYMQxlpqDGxLRvbtMhZFrZlkzMtcnYuqS2bnGXj2Glt5XBsG8fO4VhJnbcd8naOvJ3DSduOnaOQy1HI5SnYDvmcQ8F2+PKD3+TGv/4AjpXDVCFeGGRzkr1hhBAbzJRS6q6R/k1a65uGHaWUCdwNXAG8T2t958gxG/g54KSXm2yYIKOUKgOfBH5Va91efTx98DcBPOPSa9bbrHSs4jim6w3oun26bp9OWnfdPj13QNcb0HMH9LwBPc9NandAzx/Q99y07dL3BgzSkDLwPLzQP+M5OVbyBJmz7aS2bBzLJpc+oZp7u1SwyGFiU8LGxMHEwsDGxB5pr65NDKyRYmZFrWgbI2MKlY6ptLe6Ju0BK/4/2ShwbMVm1YhGEx/XTuoYTZT2I2IiNHFaJ/2kHRITExOuKFHWDogIdUwQRoRhjO9F2XGPiCAtHj4BA9wryrT6XfxwJFCuCpZnyzQMKvki7/2Hm/nwbZ+l6OQpOnlKuQKlfIGSU6Cc1kUnT9lJxsv5IuV8kUpal/NFyk5hrCtOQojxi5t9ep+752zvZkFrff16B7XWEfB0pVQd+LRS6lqt9X3p4fcDt2utv3yyH7IhgkyavD4J/IXW+lPjmIMX+LQHPVr9Lq1+h2a/S3vQS0rW7tIZ9Gm7PTqDPp1Bj85Iu+sNTvnnFXJO9qRScgoUc3mcxwbMkqNIjQI2RXIUsFeUPFZaJ+21agcLJw0nRqiSq5BOfWriXNt74sMxGp8IjxCPEJcAd6Q+TIv/zu3sZYEf4jKez9UERAwIstKPfQbtgH47oE+b5mUBh5bn6XlpSE7D8akqOwUqhRKVQpFKvpS1q/kS1ULSrxZKVIvlpC6UqBfL1IoVaumYY+fO8hcnhLgYaK2bSqlbgRcC9ymlbgSmgdeeyu3HHmRUcoHDnwDf01q/+2zuyw08mr0Oy70Oy702y702zV43rTs0+0lp9btZPwkuXdzgxK+KlVJU8+kf8/QPeeWxgB1UKNOggkMJhzK5rC7jUCJHKW0X03YRG9M3wAc6Z/OIxVZgoNIQevw/x6/wODfyD/TxeR+v4F9y7and6WPHD0XE9Ano4dPHp4tHd1W7N6w9j47n0W36dPAYXOEy11rMwn3H7WfXL60nb+eoFcvUimXqxQr1UoV6GnSG/YlSlYnSyvZEqSohSIgtTik1DQRpiCkAzwd+Tyn1i8ALgB/X+tSuZxh7kAGeQ/I+2L1KqW+lY28bveBntQNLR3n1+29kqddiudtmqdtiudeh77vr/hCl1LE/oMUKlScCrqJMhQY1CtTIU11RHCojdUnnMAZKVjbEefVp7uVdfJFDtKiQp43LFUzxcV7NVUyf1X2bGFRwqOCc/OTVVq0qxeg0/Hi08Wjj0sGlhUsn7bcCl1ZrQLvl0aLP3J6Qhw7vy15AnCgIFXN5JsvVJNyUq0yWajQqNRrlpExV6jQqdRqVtF2uSfgRYnPZDnw0vU7GAD6utf5bpVQIPAF8Nf0gx6e01r9zojsae5DRWt8BnNbGF8u9DvcfeITJcpVt+w2ezE4mKFBfVWrk07pARTsYPQW98/RAhDhLn+Ze3sLfMCC5gLeNi4nitTz7rEPMuWagstC/41Rv9MSxZoymg0eLAc20tHBpMmCZPk1/QHPJZXmpT/sKn/sPPMJit8Vyr71uAKoVykxV60xV6kxXJpiqJvV09ViZqU4wXZ1kolSR/XaEGCOt9XdIPtyzevy0c8nYg8yZ+GfxDH9/5DXjnoYQ59S7+GIWYoYiNO/hdn6G68Y0q/PDQFEjT408lzBx4pNHVoMiYlq4LNJjkT6L9FimzwI9Fgd9lgZ9lidzPDp3gK8/cj8Lneaan7azTJOZ6mRSahPMVCeZrTWYrU0yU5tktpb0t9UblJzCOX70QohzaVMGGSG2ooO01hw/tM74xcjEYJIikxS5cr2TvnesGaNpMmCeLgv0snoh6jG/3GV+ucfRPUvcu28vc+1lovj4vXnKToHZeoNttQaz9Qbb61NsS/vb6lNpf4pKoXheHrMQ4sQkyAgxZj4Rv8s/rnt8B7ULOJutxUBlwefq9U5K3/Iahp6jdJinyxxdjtJhzusyf7TLYg2+/cRDfP5bX1nzeryyU2D7xDTbJ6bYUU/q7fUpdkxMs2Nymp0T00xXJzAN87w9XiEuRhJkhBijI3R4PX/NN9jP87icr7FvxdtLBWx+nR8f4wwvHqOh58mssenmQ0ml0XTxOUonK0focNTrcORIh/mSxz899C0ONxeO233ZMk121KfZOTnDrsYMuxvb2DU5y+6RtqzsCHF6JMgIMSZ38gSv5xN08Xg/r+AlXLviU0s7qPHr/Dgv46njnqoYoVDZp7+uYOr4Ex5JqhjNIj2O0OEwbY7Q5lDU5tBim6MNxdf33s+nv3HrcWGnXqqwu7GNPY1t7G7McsnUNnY3tnHJ1DYumdrORKlyAR6lEJuHBBkhLjCN5k+4k//CLeymzsf4Oa4m+Y6kl/FUCS5bhIFimjLTlHkq21ceTFd3ImLm6HKQFgdocpAWB3stDvRa7A33c+t376K3aqPNWqHMJVPb2DO1Pav3TG/Park4WVxsJMgIcZ6NrrJsp8p2qtzNAV7A1fx3XkqV/LinKMbExMj+m7ie3SsPHkpC7zIDDtBkH00O0mTfoMmB/U32Rvv5P/d/47jrdWaqk1w6nYSaS6d2cOn0Di6b2cFlMzvZVmvI10+ILUeCjBDn0eq9YQ7R5hBtXsxTeD+vxDi9LZTERUaNXLfzfat37EmDziJ99rHMfprsY5kn2ssc2qm4c+99fPLO/7Pi4+d5O8el00m4edLMzqTM7uKKbbvZNTkjFyKLTUmCjBDn0Vp7wwB8k4MSYsRZUyimKDFFievYdexA+hH0gIgDtHiCpSTkBMs8fmiZ/cYRvvzAN1e8beVYOS6b2cEVs7u5Ytturtx2CVdtv4Qrtl3CZLl6gR+ZEKdOgowQ59F6e8DI3jDiQrAxuYxJLmNy5YEDyWrOUbo8zhKPssij4SKPHVrkIf0E//CdrxJEYXZ6o1zjym2XcOX2JNxctW0PV+3Yw56pbbKKI8ZOgowQ50kHDwcLl/C4Y7I3jBg3hWIbFbZR4VnsOXbgMITE7GOZR1nkERZ4pLvIE0bE57/9Ff7sy3+XnZqzbK6Y3cWV2/dwzY5LuWr7Hq7evocrt+8mb5/Bd3oJcQYkyAhxHjzCAr/AX+ERYmMQcOw6BdkbRmx0FgZPosGTaPATXJUMpp+0WmbAoyzwMAvsDRfYe3CBe4OH+Zu7b8+uxzGUwZ7p7VyzYw9Xb7+Ua3ZcyjU7L+Pq7XsoOnJxuzi3JMgIcY59gYd4A5/CxuSveDVH6cjeMGLLmKDA97Ob7x/9lNUcuIQ8yiJ7mechPc/euQUesQ/zhXu/nr1NpZTiksY2nrzzMp6y60lcu/tyrt11OZfP7sIy5elInJlT+i9HKXUNsBO4U2vdHRl/odb68+drckJsJjGa/8Ht/AG38VS288f8a3ambyFJcBFbXR6LpzDLU0Z3RT6YXHD8OEs8lAachxfmedg5zBfuuzPbDDBv57hmx2VJsNl9OU+95Equ3X059aJs/idO7qRB5v+2d+/hcdX3ncffX11GGl0tS5Ys2bJlybZs2cY2toEA4VJIStMADQmbS5M0oVn2km6bJs+2SbNb2k2f3TTtdsOzbXfrBkqzScgNSNKWlhBCw0PAEDAQbMzFxjdZtmVZd2lGI818948ZKbIt27KwdOZoPq/nOc+ZM5ej73l8mY/O7/y+x8x+G/gk6evg7zGz33H372de/u+AgozkpNP7w1RTyssc5TYu4U95N1EKgy5RJHCF5LOKRaxiEb86/uQRGGGMvXSxh+O8MnqM10vH+JeXnuJrTz488dll1YvZsGwl6xtXsqFxJRuWrWR5TT1mmvEnvzCdMzL/Ftji7oNm1gR818ya3P1u0PxRyU1n6w9zGxu4m1/D9E9D5JyKKGAdi1nHYmAjvJKeSdXJIK9wnN0cY8/J4+yJHOLhF3+KuwNQWVLGJctWnbKsrl+moakcNp0/+bzx4SR3P2Bm15EOM8tRkJEcdbb+MM9wSCFGZIYMo45y6i92i/QAAB9ISURBVCjnelamnzwKwyR4lU52c4zdw8fYMzLMPY9/j/hoAkgPTa1vXMnG5aszyyrWNqygqDAS4NHIXJlOkDluZpvc/UWAzJmZdwP3ggb+JTepP4zI3CkhwqUs/UXTv/3pKeL76GIXx9g9epQ9kRG+s+NR7nn8e0B6avj6xha2NrexZcVatja30VK3VMNS89B0gsxH4dRGGO4+BnzUzP5mVqoSyWIxRimmcMozMuoPIzI3CsijlVpaqeW9XAKvpi+4P0gPuzjKS2Md7IoM8/Un/5ntjz0IpO8sPh5qtjW3saW5TV2L54HzBhl3bwcws7uBT/n4QGX6tZ/OYm0iWaeDfj7BN4kxSgF5jKk/jEjWyMMmOhnfzDp4LX2H8dc5wQsc4YWhdl7uPcmf/cNXJ3rerKxrZFtLG1tb1rGtuY11S5t1vU3IXMif1gDwAzP7gLsPmdkvA3/o7lfNUm0iWeV5DvMJvsUwo/wdH2CQhPrDiGS5fPJYSx1rqeNDXArtMESCl+hgJ+28cLydH8We5f6nHgGgJFLMlua1bGtZx+Ut69nW0kZ1+YKAj0LOZdpBxt3/i5l9CPhXM0sAg8BnZ60ykSzyHV7k9/lH6qngm3yUVmoB9YcRCaNSIlxJE1fSBID3O4fp5Xna2Zlo58V4P3f/8/0kU+k+Ny11S7l85Xoua1nP1a2bWLm4UdfaZJFpBxkzu4H0VOwhoB64w91fm63CRIIyuT9MA5WspobH2cfVrOD/cDtVRIMuUUQuIsNYRhXLqEr/cnIgfS3cS3TwHIfZebydR4ae5hs/TbdNq61YyFWtG7mqdRNXt25iTUOTgk2ALmRo6fPAf3X3J81sA/AtM/u0u/94lmoTmXOn94c5Qh9H6OMamvl7PkQhutOvSC6IUsgVLJ+4oaYPOm9ykh0cZEf/QZ7du4uHfvY4ADXlC7i6dTNXr9nENWs2s7p+uYLNHLqQoaVfmvT4ZTP7FeAB4MrZKEwkCGfrD7OPkwoxIjnMMFqooYUafp0teI9ziF52cICnBw6y483dfO+5dLBZVFHF1a2bePuaS7l27aWa9j3LZnxptrsfzQw3icwb6g8jItNhGMupYjlVvJ/NeHc62DzNAZ7qP8COSWdsGqoW8fY1m7lmzaVcs/ZSltUsDrj6+eUtzTFz99jFKkQkG1QSpZcz/1qrP4yInMvkYPMBNuM9zn66eYoDPNWzn8d2Pcu3nv4hAE2LGriubQs3rr+ca9u2UBEtDbj6cNNkeRHSvSa+wA/pJUYeRoqJdknqDyMiF8wwmqmmmWo+zBZ8wHmdE/yU/fz0xH4eeOYx7vvJP1CQn88VKzdw44bLuXH95VzpV2sY6gIpyEjOG2SET/IAj/EGn+ByNlDPl3hc/WFE5KIxbKIT8R1czmg8yXMc5l+Te3li+Dh/9N2/4U8e+gpdn7qNykqdAb4QCjKS09rp5WPczxuc4H/wq3yErQC8l40BVyYi81kh+byNJt5GE587DMcYYFfyqELMDCjISE6Z3COmhlLijGIY/49f5xpagi5PRHLUYspZTHnQZYSSgozkjNN7xJxgCAP+gBsVYkREQiov6AJE5spUPWIcuI+fBVOQiIi8ZQoykjPUI0ZEZP5RkJGc0E+cyFlGUtUjRkQkvBRkZN7roJ/b+DsSjJ1xmwH1iBERCTdd7Cvz2isc4yN8gyFG+AYfoYuhU+5srR4xIiLhpiAj89ZP2Me/49uUUcSD3EEbdQAKLiIi80hWBBkzuxd4N9Dp7uuDrkfCaXKPmEqi9BFjDXX8PR+igYqgyxMRkQwzawS+CtSRnkC63d3vnvT6Z4A/Bxa5e9e59pUt18jcB9wUdBESXuM9Yo7QhwO9xDCMj3OZQoyISPYZAz7j7m3AFcAnzawNJkLOO4FD09lRVgQZd38C6A66DgmvqXrEpHDu5omAKhIRkbNx96PuvjPzeADYAyzJvPy/gN+DSXfvPYesCDLTYWZ3mtlzZvbcSYaDLkeyjHrEiIhknZrx7+3McudUbzKzJmAz8IyZ3QoccfeXpvtDsuIamelw9+3AdoCN1jCtlCa5oZcYheSTIHnGa+oRIyJy4Xx0jGTHOS9NmY4ud996rjeYWRnwAPAp0sNNf0B6WGnaQnNGRmQqR+nnvfwdSVJE1CNGRCQ0zKyQdIj5urs/CLQAK4CXzOwAsBTYaWaLz7Wf0JyRETndPrr4EF+jlxjf4CN0MqgeMSIiIWBmBtwD7HH3vwBw95eB2knvOQBsPd+spawIMmZ2P3Ad6fG0duAud78n2Kok20yeXl1DGcOMECXCd/kYG6gH1CNGRCQkrgI+ArxsZi9mnvsDd3/4QneUFUHG3T8YdA2S3canV4/PTDrBIAZ8musnQoyIiISDuz8J2Hne0zSdfekaGQmFqaZXO3AvzwRTkIiIZAUFGQkFTa8WEZGpKMhIKFRSPOXzml4tIpLbFGQk6/0NT9FLnLzThlM1vVpERBRkJGs5zpf5CV/gUW5mHf+TW1lCJQYsoZIvcbNmKYmI5LismLUkMm7yFOtSihhkhNvZyJ9zC/nkcTsbgy5RRESyiIKMZI3Tp1gPMkI+eVxNM/k6eSgiIlPQt4NkjammWCdJ8SV+HFBFIiKS7RRkJGtoirWIiFwoBRnJCo5TQmTK1zTFWkREzkZBRgLnOH/EIwyRoOC0v5KaYi0iIueiICOBcpw/4VHu4Rk+weX8haZYi4jIBdCsJZlzU02x/hjbuItfxjBu45KgSxQRkZBQkJE5NfUUa+NSlmLnvhGqiIjIGTS0JHNq6inWzp9qirWIiMyAgozMKU2xFhGRi0lBRubU2aZSa4q1iIjMhIKMzKnPcgNRCk95TlOsRURkpnSxr8yp8anU47OWGqjks9ygKdYiIjIjCjIy597DBgUXERG5KDS0JCIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgI6FhFUVUPfBvsIqioEsREZEsoSAjoVF8SyvR96yl+ObVQZciIiJZQkFGQqPk45sy680BVyIiItlCQUZCwSqKiLytEYDIlY1YeSTgikREJBsoyEgoFN/SiieSAHgiSfEtrQFXJCIi2SArgoyZ3WRmr5nZXjP7bND1SPYp+fgm8jIX+eZVFGl4SUQk5MzsXjPrNLNdpz3/n8zsVTPbbWZfOt9+CmavxOkxs3zgr4B3AO3Az8zsB+7+SrCVyVyr+s7tRN/bNuVrPjJ2ynbkqkYaUndN+d7YA6/Qc/t3Lnp9IiJyUd0H/CXw1fEnzOx64FZgo7uPmFnt+XaSDWdkLgP2uvub7p4Avkn6ICTH9H/uMRIvHCU1mDjjNSsqOOc2QGowQWLnUfo/99is1SgiIheHuz8BdJ/29H8AvujuI5n3dJ5vP9kQZJYAhydtt2eekxyT3NtN17a/ZeAPHyc1PIqPpab1OR9LkRoeZeAPH6dr23aSe0//dyEiIiGxGni7mT1jZj8xs23n+0DgQ0vTZWZ3AncCLKEy4Gpk1qScoS/vIP6Pr1P1zfdRsLqavLKzz1BKDSYYe72Lnvd/l+S+njksVERk/sorixK9Zv1b28l91JjZc5Oe2e7u28/zqQJgIXAFsA34tpk1u7uftda3VuVFcQRonLS9NPPcKdx9u7tvdfet1ZTMWXESjPTZme0MfvFJfHh0yvf48CiDX3ySrm1/qxAjIpJ9usa/tzPL+UIMpEdlHvS0Z4EUUHOuD2RDkPkZsMrMVphZBPgA8IOAa5Js4JA81HfWISYfS5E82AtnzekiIhIy3wOuBzCz1UAE6DrXBwIPMu4+BvwW8AiwB/i2u+8OtirJFpOnXXvKSQ0l8FQ6uWgatohIeJnZ/cDTQKuZtZvZbwL3As2ZKdnfBH7jXMNKkCXXyLj7w8DDQdch2WVyN18fGSPVE6fvU/9C5ZdvIq+qGCsqmOjy6wNnznQSEZHs5e4fPMtLH76Q/QR+RkbkbIpvacWTKVJDCUaf76Bz/V8T//ZuOtf/NaPPd2TOzqTU5VdEJIcpyEjWKvn4JvJKIwx/ZSdd196Hd8cA8O4YXdfex/BXdpJXEtHwkohIDsuKoSWRqSQ7h+j+4HeJf2uKS6aSTv/vPkJiRzvFt66Z++JERCQrKMhI1ur94APnfU/8W7unDjoiIpITNLQkIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioaUgIyIiIqGlICMiIiKhpSAjIiIioRVokDGz281st5mlzGxrkLWIiIjI3DGz381kgF1mdr+ZFc9kP0GfkdkF3AY8EXAdIiIiMkfMbAnw28BWd18P5AMfmMm+Ci5mYRfK3fcAmFmQZYiIiMjcKwCiZjYKlAAdM9lJ0Gdkps3M7jSz58zsuZMMB12OiIiInFvN+Pd2Zrlz/AV3PwL8OXAIOAr0ufsPZ/JDZv2MjJn9CFg8xUufd/fvT3c/7r4d2A6w0Rr8IpUnIiIip0kd7Gfgjsfe6m663H3K61/NrAq4FVgB9ALfMbMPu/vXLvSHzHqQcfcbZ/tniIiISKjcCOx39xMAZvYgcCVwwUEmNENLIiIiMm8cAq4wsxJLXyh7A7BnJjsKevr1e8ysHXgb8E9m9kiQ9YiIiMjsc/dngO8CO4GXSeeR7TPZV9Czlh4CHgqyBhEREZl77n4XcNdb3Y+GlkRERCS0FGREREQktBRkREREJLQUZERERCS0FGREREQktBRkREREAuY4R+kPuoxQCnT6tYiISK4aZIQn2c+/spcnqo9wcrCP7sQXiUQiQZcWKgoyIiIicyBJipc5yk/Zz5NrutnxxsuMJscoK4py3bKt3LDhMpLJZNBlho6CjIiIyCxI4bxKJ0+xn2c2DfDUaz+nLzYIwLrBZv7jO2/nHRuu4PKV64kUFAIQjUaDLDmUFGREREQuAsd5gy6eYj9Pc5AdZenhIoDmjiW857LruWbtpby9dTO1lQsDrnb+UJARERGZAcfZSxdPc4CnOMCO8g66BnoBaKyu452tV3Dt2i28fe1mli6sC7ja+UtBRkREZBpSOK/RyQ4OsoODPFtxlBP9PQAsqVrEjWsu5+o1m3j7ms0sr6knfVNnmW0KMiIiIlMYJsGLdLCTwzxPO8+XHac7M1TUWF3HDasv46rWjVzVuonm2iUKLgFRkBERkZznOIfpTQcWDvPC8j52Hd5HMpWeRbSyrpF3rbpqIrgsr6kPuGIZpyAjIiI5Z4gEL9HBTtrZSTsvVHRODBOVFkW5NLqG333Xh7isZT3bWtpYWFYZcMVyNgoyIiIyryVJ8QZdvEA7L3KEnzf2s7v9TVKeAtJnW25suYytLevY1tzGuqXNFOTr6zEs9CclIiLzxnir/5fo4AWO8PKaYV7Y/yqDIzEAKkvK2FKxlv9880fZ1tzGlua1OtsScgoyIiISSimcg/Swm6O8zFH2rEvw0sHXJ3q3FOTnsyG+ig9edRNbVqxla3MbLXVLycvTbQbnEwUZERHJegmSvMEJdnOMXRzl1dVxXj60l4H4MACF+QWsHVjBuzZfzcblq9m0fDUblq2kuLAo4MpltinIiIhIVuklxisc5xWO8QrH2bNsiNc6DpAYGwWgJFLM+mQL73/bO7lk+WouWbaKtiUrKCrUzRZzkYKMiIgEIkmKg/TwCsfYw3H20Mmr1X0cOnls4j2LKqrYUL6S6298HxuXreKS5atoqVtKfl5+gJVLNlGQERGRWeU4xxjgdU7wBid4jU5eXxFnz5H9DCfiAOTn5bNqcSPblq7jjutvZUPjSjYsW0ldZXXA1Uu2U5AREZGLIoXTQR+vc4K9dPEGJ9jXkuC1joMTd30GWFhWybqiZj527c2sW9rC+mUtrGlo0vUsMiMKMiIickESJDlAN3vpYh9dvEEX+5bHeePooYkzLADVZZWsLVjB+664kbUNTaxZsoI1DU0sqqgKsHqZbxRkRERkSt0Ms48u9nFyYv1m3TD7T3RMtO6H9A0TV5ct5zeueTerG5azpqGJ1vrlVJcvCLB6yRUKMiIiOSzGKAfp4U1OTiwHWkZ549hheob6J94XKSikpW4pbYub+bVt19Fav5xV9ctZWddIebQkwCOQXKcgIyIyz/UT5xA9HKCHg3RziB7a1xpvdrbT3t2Ju0+8t65yIasKlnHr1mtZXb+cVYsbWbl4GctrFmumkGQlBRkRkZAbI8VR+jlEDwfp4TA9HKKX9hVJ9p/ooDvT6XZcdVklK0aWcOXqjTTXLmHl4mWsXLyU5tqlVERLAzoKkZlRkBERyXIpnE4GaaeXQ/RwmF4O00vHWuPgiaMc6elkLPmLa1YK8vNZurCO5cX13LLlGlbULmHFogZW1C6haVGDworMKwoyIiIBGyXJMQY4Qh/t9HKEPg7Ty7G2fA51HaW9u3Oiq+24usqFLEvUs61lHe+tuYGmRQ00LaqnaVEDS6oW6e7NkjP0N11EZBalcLoZ5hj9dNBPB3100M8R+ji+0jh88hhHe0+S8tQpn6urXEhjfDGbmlq5ecs1NFYvpqmmnmU1i1lWU080op4rIqAgIyIyYzFGOc4AxxngGAMcoz+zHqBrVT4dPSc42tt1xtmUwvwCliyspbGgjmvbtrB0YR2N1XUsra6jcWEdjTV1ag4nMk0KMiIikzjOEAk6GeQ4A3QyyIlJj7vWFnKst4tjvSdP6VY7riRSTH1VDfV5NVy2ch0NVYtYUlVLQ9UiGqpqWFpdx6LyKvLy8gI4OpH5R0FGRHJCjFFOMMgJhjjJUOZxeulkiO6V+XT299DZ183QSOyMz0cKCllcWc3iRDWtDU1c27aFxQtqqF9QTV1lNQ1Vi6ivqqEyWoaZBXCEIrlJQUZEQilBkpMM0cMwJxmmKxNQ0stw+rWWAjr7e+jq72FwinACUFVaQV3lQmoLF7K1eS21FQupq6xmcSag1FWmt6tKyxVQRLJQKIPMfrr5NN+niihVRFlAlAWUTHqcXkooxNB/PCLZbHwop484vcToJUYPMXoZzqzT290M09tcwMnBPk4O9NIfG5pyf/l5+dSUV7KovIrqSBnbWtqorVhITfkCaisWUltZRU15FbWVVdRWLCRSUDjHRywiF1Mog8xYBJ4q66B7sP+UG5SdriA/n6rSCqpKy1lQUk7pvhEqiVJJMQsoppIoFRRTQTGVmfX4Uk4RBWgMW2Q6xkgxQJw+4gwwQn/mcXodo584vZnn+ogx2FJE79AAvcMD9Az1n9ID5XQlkWIWllWwsKyShcUlNC1qoKa8kpryBSwsW0BNeSXV5QuoraiipnwBC0rKdf2JSA4JNMiY2Z8BNwMJYB/wcXfvPd/nWuub+MldfwtAfHSE7sF+eob66R7sp3co/R9jT2bdOzxAz2DmP8zlY+wfPknv8AB9w4OntOWeSmlRlIpoaXopKaNkX5xyiiaWstPWpRRRRmRiXUYRpUQoCmdelHnOcUZIMkyCQUYYJMHQpPUAIwyeth5fhpqL6I8NMhAbpj82NOU1JZOZGZUlZSwoSf9SsaAoypKqWqrKyqkqrWBBSTlVpZnHpeXp4FJaSVVZuWbviMxTZnYTcDeQD3zF3b84k/0E/Q37KPA5dx8zsz8FPgf8/oXsoLiwKDMbYNEF/eBUKkV/fIjeoQH6Y0P0x4boywSc/uEh+mKD9A8P0pd5rT82SP+KFB2xQQbjnQzEhs465n66wvwCSoujlBVFKS2KUlocpXh/nFIilBAhSiElFFJCZGIdnbQ+fSmmgOJJa505mr/GSBFnlDhjE+sYo2cswyQm1sOT1uPPDZEgvqKYoXiMoZEYgyMxhuIxRpNj06qjrChKebSU8mgJFdEyyqMlLFlYS0W0lMpM0K+MllFZUkZFSSmVJeVURscfl1FRXKqzJCIywczygb8C3gG0Az8zsx+4+ysXuq9Ag4y7/3DS5g7gfXP1s/Py8iZ+O5ypZCrJYDzGQHyIgdgwA7EhBuLDDMaHGYzHGIwPT2wPZb44xr9AhldHOBaPMTzSy/BIPP36SJxk6uyn2M+mID+faGERxZEiooVFFBVGKC6MUFQYoeDNIYoomFgi5GeWAooy6/HnCjNLhAIKyZvYLsg8nrwumFj/YsmfWOyUx3mTnjMs85xltmb/GibHSWWWJI7jJEmRxEll1untUx+PnbIkJx6PZh6PrxMkGSPJKCkSjDFKklGSjGTWCcYYyawTJBmZtB5hjLHmUkZGE8RHE4yMJoiNjhBPjBAbHTnnkMvZ5OflU1YUpaSoOLNEKS+uoLooyrLqekqLiiktTofqsuISyotLKCsuoaz4F9vjoaW8uJSy4qhuFigiF9tlwF53fxPAzL4J3AqEK8ic5g7gW0EXcSHy8/KpLEn/FnoxuDuJsVGGRuIMJ2LEEwmGE3FiiThDI+l1bGTklC+6WCK9jG8nxkaJjyaIJ0bSX4yrCxkcTdA9Nkp8NMbIWILE6CgjY6OMjCZIJEdn9GV5MeVZHmZGnhlm6XBjxsQMkfGwM3nGyPiwoOMT2+6Z0OKp9GP3M7qlzrWC/Hwi+YUUFUYoKigkUlhIUUE6aEYKIhQXllFSGCFauXAigEYKCieCaUmkmOJI5JSgWlJUTDRSTElRUXqdeU9JJEppUTGRgkLNrhGRbLcEODxpux24fCY7mvUgY2Y/AhZP8dLn3f37mfd8HhgDvn6O/dwJ3JnZHKm845pdF7vWEKkBuoIu4mJJeQocphmnQnXsY8kkY8nkOS9Kv0ChOv5ZoOPX8c/v47/jrK+0zmEV/Jyjjyzlj2ve4m6Kzey5Sdvb3X37W9znGWY9yLj7jed63cw+BrwbuMHPcfVt5uC3Zz7znLtvvZh1hkkuH38uHzvo+HX8Ov5cPf7TAsGsc/ebZvlHHAEaJ20vzTx3wQK9+i5zxfLvAbe4+3CQtYiIiMic+RmwysxWmFkE+ADwg5nsKOhrZP4SKAIezYzp73D3fx9sSSIiIjKbMrOVfwt4hPT063vdffdM9hX0rKWVM/zoRR9jC5lcPv5cPnbQ8ev4c1suH/+8O3Z3fxh4+K3ux87XFE5EREQkW6lDlYiIiIRWaIOMmf2Zmb1qZj83s4fMbEHQNc0VM7vdzHabWcrMcuYKfjO7ycxeM7O9ZvbZoOuZS2Z2r5l1mllOth0ws0Yze9zMXsn83f+doGuaK2ZWbGbPmtlLmWP/46BrCoKZ5ZvZC2b2j0HXMtfM7ICZvWxmL8717KUwCG2QIX17g/XufgnwOunbG+SKXcBtwBNBFzJXJrWz/hWgDfigmbUFW9Wcug+Y7emQ2WwM+Iy7twFXAJ/MoT//EeCX3H0jsAm4ycyuCLimIPwOsCfoIgJ0vbtvytXp5+cS2iDj7j909/EbxewgPQc9J7j7Hnd/Leg65thEO2t3TwDj7axzgrs/AXQHXUdQ3P2ou+/MPB4g/YW2JNiq5oanDWY2CzNLTl3caGZLgV8FvhJ0LZJ9QhtkTnMH8M9BFyGzaqp21jnxRSanMrMmYDPwTLCVzJ3MsMqLQCfwqLvnzLFnfJl0z7Fg7zkSHAd+aGbPZ7rcyyRB95E5p4t1e4Mwms6xi+QaMysDHgA+5e79QdczV9w9CWzKXAv4kJmtd/ecuF7KzN4NdLr782Z2XdD1BORqdz9iZrWk+669mjlLK2R5kLlYtzcIo/Mdew66aO2sJZzMrJB0iPm6uz8YdD1BcPdeM3uc9PVSORFkgKuAW8zsXUAxUGFmX3P3Dwdc15xx9yOZdaeZPUR6qF1BJiO0Q0u6vUHOuWjtrCV8LN36+x5gj7v/RdD1zCUzWzQ+K9PMosA7gFeDrWruuPvn3H2puzeR/nf/41wKMWZWambl44+Bd5I7IXZaQhtkSN/eoJz0abYXzez/Bl3QXDGz95hZO/A24J/M7JGga5ptmQu7x9tZ7wG+PdN21mFkZvcDTwOtZtZuZr8ZdE1z7CrgI8AvZf69v5j5DT0X1AOPm9nPSQf6R90956Yg57A64Ekzewl4Fvgnd/+XgGvKKursKyIiIqEV5jMyIiIikuMUZERERCS0FGREREQktBRkREREJLQUZERERCS0FGREREQktBRkREREJLQUZETkgpjZvWbWaWbqLioigVOQEZELdR/pe/2IiAROQUZEpmRmj5vZOzKP/8TM/jdA5q673YEWJyKSkdV3vxaRQN0F/DczqwU2A7cEXI+IyBkUZERkSu7+ROau058GrnP3ZNA1iYicTkNLIjIlM9tA+s7LCXcfCLoeEZGpKMiIyBnMrB74OnArMGhmurhXRLKSgoyInMLMSoAHgc+4+x7gC6Svlxl//X7gaaDVzNrN7DeDqVREBMzdg65BREREZEZ0RkZERERCS0FGREREQktBRkREREJLQUZERERCS0FGREREQktBRkREREJLQUZERERCS0FGREREQuv/AytENDXXh9ctAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "w = paddle.to_tensor([0.2, 2])\n",
    "model = OptimizedFunction(w)\n",
    "opt = Adam(init_lr=0.2, model=model, beta1=0.9, beta2=0.99, epsilon=1e-7)\n",
    "train_and_plot_f(model, opt, epoch=20, fig_name='opti-vis-para5.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，Adam算法可以自适应调整学习率，参数更新更加平稳。\n",
    "\n",
    "**简单拟合实验**  训练单层线性网络，进行简单的拟合实验。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAEICAYAAAB74HFBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XucZFV97/3Pty59n/s0w8xwGy6iI8qAI0o0PoiXIBoxOZ4IL4+aIyfEqHn0JMZ4yaOYRKMnUZ9jNPFB4IAegxAV5ShRiOJBjgoOOMAAIjAwMPee+6V7uuvye/6o3TNFT/d0dXd1167q7/v1qlftvfauvX9VXb3612uvvZYiAjMzMzObnEyjAzAzMzNrZk6mzMzMzKbAyZSZmZnZFDiZMjMzM5sCJ1NmZmZmU+BkyszMzGwKnEy1GEkPSbqggec/SdIBSdlpOHZIOr3exzUzG0nSKUmdk6tx/+sk/e10x2Xp5GSqxUTE8yPiJwCSrpT0P6fzfJKekvTqqvM/HRE9EVGazvOamZmlhZMpG1Ot/5GZmZnNZk6mWsxwS5Gki4CPAG9JLrvdn2yfJ+kaSVskbZL0t8OX5CT9oaT/I+nzknYCV0o6TdKPJe2UtEPS1yXNT/b/GnAS8L+Sc3xwZNO4pGWSbpG0S9Ljkv6oKtYrJd0k6auS9ieXKFfX+D7nJa/rk7RB0l9JyiTbTpf0vyXtTWK+MSlX8t62S9on6UFJZ9XtwzezaZPUJd9KfueflPR/V227UtI3Jd2Y1CX3STq7avvzJP1E0p6knnlj1bZOSZ9N6pG9ku6S1Fl16rdKejqpSz46gXj/KKnzdiV14LKkfMx6SNLFkh5O3sMmSR+Y0odmM8bJVIuKiB8AnwJuTC67DVcs1wFF4HTgHOC1wH+peulLgPXAEuCTgIC/A5YBzwNOBK5MzvE24Gngd5Nz/LdRQvkGsDF5/ZuBT0m6sGr7G5N95gO3AF+s8S3+IzAPOBX4v4C3A/852fY3wG3AAuCEZF+S9/oK4DnJa/8A2Fnj+cysQZJ/lP4XcD+wHHgV8H5Jv1O12yXAvwILgX8BviMpLymfvPY24DjgT4GvSzozed0/AC8Cfit57QeBctVxXw6cmZzzY5KeV0O8F1KpN/8AWApsoFLPwbHroWuAP46IOcBZwI/HO5elg5OpWUTSEuBi4P0RcTAitgOfBy6t2m1zRPxjRBQjYiAiHo+I2yNiMCL6gM9RSV5qOd+JwMuAv4yIQxGxFriaSuIz7K6IuDXpY/U14OxRDjXyuNkk5g9HxP6IeAr4LPC2ZJcCcDKwLDnvXVXlc4DnAoqIRyJiSy3vxcwa6sVAb0T8dUQMRcR64Cs8u+66NyK+GREFKvVUB/DS5NEDfDp57Y+B7wGXJUnaO4H3RcSmiChFxM8iYrDquJ9I6sL7qSRz49ZRwFuBayPivuRYHwbOl3QKx66HCsBKSXMjYndE3DfhT8oawsnU7HIykAe2JM3de4D/j8p/a8OeqX6BpCWSvpE0Oe8D/iewuMbzLQN2RcT+qrINVP6zHLa1arkf6Kihr9bi5H1sGOO4H6TSonZP0qT/ToCkEv0i8CVgu6SrJM2t8b2YWeOcDCwbrreSuusjVFrQhx2uuyKizJEW8WXAM0nZsOH6YjGVpOuJY5x7ZB3VU0O8y6iqnyLiAJXWp+Xj1EP/gco/vBuSrgrn13AuSwEnU60tRqw/AwwCiyNifvKYGxHPP8ZrPpWUvSAi5gL/iUqiMtb+1TYDCyXNqSo7Cdg0kTcxih0caX066rgRsTUi/igilgF/DPyTkiEVIuILEfEiYCWVZva/mGIsZjb9ngGerKq35kfEnIi4uGqfE4cXkhanE6jUQZuBE4f7VCaG64sdwCHgtDrHu5mq+klSN7CII3XUqPVQRPwyIi6h8g/ud4Cb6hyXTRMnU61tG3DKcCWSNCXfBnxW0lxJGVU6mB/rst0c4ACwV9Jyjk4+tlHpt3SUiHgG+Bnwd5I6JL0QuJxK69akJZcEbwI+KWmOpJOBPxs+rqT/KOmEZPfdVBK+sqQXS3pJ0ofiIJVKtHz0GcwsZe4B9kv6y6TDeFbSWZJeXLXPiyT9ftKy/X4q/zj+AribSovSB5M+VBcAvwt8I2mtuhb4XNLBPSvpfEntU4z3BuA/S1qVHOtTwN0R8dRY9ZCkNklvlTQvuVS5D9dPTcPJVGv71+R5p6Tha+9vB9qAh6kkGt+k0kFyLJ8AzgX2At8Hvj1i+98Bf5U0vY9258llwClU/lO7Gfh4RPz7xN/KUf6USkW0HriLSofTa5NtLwbulnSASqf29yV9LOZS6Wexm0oT/E7g7+sQi5lNo+QfqDcAq4AnqbQoXU2lA/ew7wJvofL7/Tbg9yOiEBFDVJKn1yWv+yfg7RHx6+R1HwAeBH4J7AI+wxT/NiZ13P8DfAvYQqXla7h/17HqobcBTyVdKt5Fpe+VNQFFHOsqjZmZWbpJuhI4PSL+U6NjsdnJLVNmZmZmU+BkyszMzGwKfJnPzMzMbArcMmVmZmY2BTM6ke3ixYvjlFNOmclTmlmD3XvvvTsiorfRcUyV6y+z2afW+mtGk6lTTjmFNWvWzOQpzazBJG0Yf6/0c/1lNvvUWn/5Mp+ZmZnZFDiZMjMzM5sCJ1NmZmZmUzBuMpXMqXaPpPslPSTpE0n5dZKelLQ2eaya/nDNzMzM0qWWDuiDwIURcSCZmPEuSf+WbPuLiPjm9IVnZmZmlm7jJlNRGdXzQLKaTx4e6dPMzMyMGvtMScpKWgtsB26PiLuTTZ+U9ICkz0tqH+O1V0haI2lNX19fncI2MzMzS4eakqmIKEXEKuAE4DxJZwEfBp4LvBhYCPzlGK+9KiJWR8Tq3t7axu0rPr6TfR/7McWn99S0v5lZmgw++AyFp3c2OgwzmyETupsvIvYAdwAXRcSWqBgE/gdwXr2CKq7fzf6/uZPSxn31OqSZ2YwZengzxQ1Opsxmi1ru5uuVND9Z7gReA/xa0tKkTMCbgHV1i0qqPJfdNcvMmk+mu53ywcFGh2FmM6SWu/mWAtdLylJJvm6KiO9J+rGkXkDAWuBddYsqkyRTzqXMrAmpu53y7oONDsPMZkgtd/M9AJwzSvmF0xIRRxqm3DJlZs0o091OceMuIgIdrtDMrFWlcwT0wy1TTqbMrPmoqw2KZRgqNToUM5sB6U6m3DJlZk0o010ZKcb9psxmh3QmU0mzeDiZMrMmpOFkqt/JlNlskM5kyh3QzayJZboqyVS4ZcpsVkhnMuUO6GbWxNSVB0G5f6jRoZjZDEhlMiV3QDezJqZMBnW2uWXKbJZIZTLlQTvNrNl54E6z2aOWQTtnnvtMmdk4JF0LvAHYHhFnJWU3Amcmu8wH9iTzio587VPAfqAEFCNidd3j62qjvKe/3oc1sxRKZzI1nEu5ZcrMxnYd8EXgq8MFEfGW4WVJnwX2HuP1r4yIHdMVXKa7neJmT9ZuNhuk8zKf+0yZ2Tgi4k5g12jbkjlD/wC4YUaDqo6hqx0KJWKo2KgQzGyGpDOZcp8pM5ua3wa2RcRjY2wP4DZJ90q6YqyDSLpC0hpJa/r6+iYUgAfuNJs9UplMySOgm9nUXMaxW6VeHhHnAq8D3iPpFaPtFBFXRcTqiFjd29s7oQDU3VY5hodHMGt5qUym3AHdzCZLUg74feDGsfaJiE3J83bgZuC8esfhlimz2SOdyZQH7TSzyXs18OuI2DjaRkndkuYMLwOvBdbVOwh1JS1TTqbMWl46k6mkZSrcAd3MxiDpBuDnwJmSNkq6PNl0KSMu8UlaJunWZHUJcJek+4F7gO9HxA/qHl8ycKdbpsxaX0qHRnCfKTM7toi4bIzyPxylbDNwcbK8Hjh7WoNLqLvNU8qYzQKpbplynykza2aZrnZf5jObBdKZTLnPlJm1gEx3O+V+J1NmrS6VyZQnOjazVqDudhgqEQUP3GnWylKZTLnPlJm1gkwy1lT5oPtNmbWycZMpSR2S7pF0v6SHJH0iKV8h6W5Jj0u6UVJb/aIavpuvbkc0M5tx6qqMNeV+U2atrZaWqUHgwog4G1gFXCTppcBngM9HxOnAbuDyYxxjglG5ZcrMmt/hgTvdb8qspY2bTEXFgWQ1nzwCuBD4ZlJ+PfCmukXlDuhm1gI8cKfZ7FBTnylJWUlrge3A7cATwJ6IGO5VuRFYPsZrJzxRqDugm1krUDaDOvPuM2XW4mpKpiKiFBGrgBOozGH13FpPMKmJQt0B3cxahDzWlFnLm9DdfBGxB7gDOB+Yn0woCpUka1P9ovKgnWbWGjzWlFnrq+Vuvl5J85PlTuA1wCNUkqo3J7u9A/hu3aIazqXcMmVmTU5dbb7MZ9biapmbbylwvaQsleTrpoj4nqSHgW9I+lvgV8A1dYvKfabMrEVkutthqEgUSiifbXQ4ZjYNxk2mIuIB4JxRytdT6T9Vf+4zZWYtonp4hOy8rgZHY2bTIZ0joLvPlJm1CHV7eASzVpfKZEoeZ8rMWkQmGQXd/abMWlcqkyn3mTKzVuEpZcxaX6qTKd/NZ2bNTrkM6sh7eASzFpbOZMod0M2shairjfBlPrOWlc5kyh3QzayFZLrbKfsyn1nLSmcy5Q7oZjYOSddK2i5pXVXZlZI2SVqbPC4e47UXSXpU0uOSPjTtsXa3E77MZ9ayUplMeaJjM6vBdcBFo5R/PiJWJY9bR25MBiD+EvA6YCVwmaSV0xloprudGCwSxdJ0nsbMGiSVyZT7TJnZeCLiTmDXJF56HvB4RKyPiCHgG8AldQ1uBHVVxpry8AhmrSmdyZT7TJnZ5L1X0gPJZcAFo2xfDjxTtb4xKTuKpCskrZG0pq+vb9IBDY+C7uERzFpTOpMpT3RsZpPzz8BpwCpgC/DZqRwsIq6KiNURsbq3t3fSx6meUsbMWk86kyn3mTKzSYiIbRFRiogy8BVGnz90E3Bi1foJSdm0Gb7M55Yps9aUzmTKfabMbBIkLa1a/T1g3Si7/RI4Q9IKSW3ApcAt0xpXLovac5T73WfKrBXlGh3AaCT3mTKzY5N0A3ABsFjSRuDjwAWSVlGpPZ4C/jjZdxlwdURcHBFFSe8FfghkgWsj4qFpj7e73S1TZi0qlckUULnU55YpMxtDRFw2SvE1Y+y7Gbi4av1W4KhhE6aTB+40a13pvMwHIHdAN7PWUZlSxsmUWStKbzKVkTugm1nLODJwZ7nRoZhZnaU3mZIv85lZ61BXMtaUh0cwaznpTaYycgd0M2sZme7hUdCdTJm1mnGTKUknSrpD0sOSHpL0vqS8pglFJ0vCLVNm1jJ0eOBOD49g1mpquZuvCPx5RNwnaQ5wr6Tbk22fj4h/mJbI3GfKzFpIpstTypi1qnGTqYjYQmVaBiJiv6RHGGMeq7qSfDefmbUM5bPQlvNlPrMWNKE+U5JOAc4B7k6KxptQdPIThbrPlJm1mEy3h0cwa0U1J1OSeoBvAe+PiH3UOKHopCcKFZR39lM+4IrHzFpDprvdfabMWlBNyZSkPJVE6usR8W2oeULRKUQmBr72ANtO/0JdD2tm1ijq8pQyZq2olrv5RGWKhkci4nNV5bVMKDppylTm5ytvO1jPw5qZNUymu504VCBKHrjTrJXUcjffy4C3AQ9KWpuUfQS4bLQJRetmeLJjM7MWoWSsqegfRHM6GxyNmdVLLXfz3QWMltlM7yShGSdTZtZaMsNjTR0cIuNkyqxlpHcEdOdSZtZi5LGmzFpSepMpt0yZWYvxlDJmrSm9yZT7TJlZi1E+B21Zyp7s2KylpDeZMjNrQZmuduKgx5oyayXpTaaKvnXYzFqPutvdMmXWYlKbTEWh1OgQzCzFkmmstktaV1X295J+nUxzdbOk+WO89ilJD0paK2nNzEWdjDXlPlNmLSW1yRQFt0yZ2TFdB1w0oux24KyIeCHwG+DDx3j9KyNiVUSsnqb4RpXpaiMGPHCnWStJbTLllikzO5aIuBPYNaLstogoJqu/AE6Y8cDGoWSsqfAcfWYtI7XJlFumzGyK3gn82xjbArhN0r2SrhjrAJKukLRG0pq+vr66BHV44E73mzJrGelNpsrR6AjMrElJ+ihQBL4+xi4vj4hzgdcB75H0itF2ioirImJ1RKzu7e2tT2zDU8q435RZy0hvMmVmNgmS/hB4A/DWiBj1v7KI2JQ8bwduBs6bqfgyXUemlDGz1uBkysxahqSLgA8Cb4yI/jH26ZY0Z3gZeC2wbrR9pyXGthy05SjvH5ipU5rZNGuKZKq0eR/lA24SN7MjJN0A/Bw4U9JGSZcDXwTmALcnwx58Odl3maThydmXAHdJuh+4B/h+RPxgJmPPLuymvOvgTJ7SzKZRrtEB1GLr8s+Re95iljz83kaHYmYpERGXjVJ8zRj7bgYuTpbXA2dPY2jjyi7sZujRrUQ5kOchNWt6TdEyBVB8ZEejQzAzq4vMwh4olSnv86U+s1bQNMmUmVmryC7qAaC860CDIzGzenAyZWY2wzLzOyEjSjvdb8qsFaQ/mcq6P4GZtRZlMmQWdFNyy5RZS0h/MlXy4J1m1nqyC7sp7zzAGENhmVkTSW0y1fayExsdgpnZtMku7CEGi56jz6wFjJtMSTpR0h2SHpb0kKT3JeULJd0u6bHkeUE9A+u963LmfvLCeh7SzCw1Mou6AXypz6wF1NIyVQT+PCJWAi+lMo/VSuBDwI8i4gzgR8l6feVS23BmZjYl2YWVZKrsTuhmTW/cbCUitkTEfcnyfuARYDlwCXB9stv1wJvqHl322eGVd406O4SZWdNRPkdmTodbpsxawISafiSdApwD3A0siYgtyaatVKZoGO01V0haI2lNX1/fxKIbcSPf9pdePbHXm5mlWGZRDyVPK2PW9GpOpiT1AN8C3h8R+6q3JTOzjzU7+1URsToiVvf29k4ouOgvPGu99NiuCb3ezCzNsgu7if2HiKFio0MxsymoKZmSlKeSSH09Ir6dFG+TtDTZvhTYXu/g4mBh/J3MzJpUJhkJ3a1TZs2tlrv5RGXy0Eci4nNVm24B3pEsvwP4br2Di4O+ZdjMWld2YZJM7XS/KbNmlqthn5cBbwMelLQ2KfsI8GngJkmXAxuAP6h3cHHAyZSZta5MVxvqzFN2y5RZUxs3mYqIuziqK/hhr6pvOCPO3e/LfGbW2jILe3xHn1mTS/VATh2/e2ajQzAzm1bZhd2U9/QTpXKjQzGzSUp1MtX11heSXTG/0WGYmU2b7MIeKAflPR5Hz6xZpTqZAlBHLd26zMyak6eVMWt+qU+myIzVXcvMrPll5nZCLuNpZcyamJMpM2tKkq6VtF3SuqqymiZgl/SOZJ/HJL1jtH1miiSy7oRu1tTSn0zJyZSZjeo64KIRZeNOwC5pIfBx4CXAecDHx0q6ZkpmYTelXQepTCZhZs0m9cmU3DJlZqOIiDuBkXNM1TIB++8At0fErojYDdzO0UnZjMou6oFCidh/qJFhmNkkpT6ZGjnClf9zM7NjqGUC9uXAM1XrG5Oyo0xpovYJyC50J3SzZpb+ZGpky1Sh1Jg4zKypHGsC9gkcY9ITtU9EZn43CEruhG7WlJoumYpDnl3dzMZUywTsm4ATq9ZPSMoaRrkMmXldlN0yZdaU0p9MjeiAHgNOpsxsTLVMwP5D4LWSFiQdz1+blDVUdlEPJc/RZ9aU0p9MuWXKzEYh6Qbg58CZkjYmk65/GniNpMeAVyfrSFot6WqAiNgF/A3wy+Tx10lZQ2UWdhP9Q5QHPMG7WbNJ/fDi7a84mcIvNjLno7/N/k/+lBjw5MdmBhFx2RibjpqAPSLWAP+lav1a4NppCm1Ssot6ACjvOkhmeVuDozGziUh9y9TcT17IcQ+9m/yLlgFumTKz1nT4jr6d7jdl1mxSn0wplyW/8rgjc/Q5mTKzFqT2POpp9/AIZk0o9cnUMHVWkil3QDezVpVd2EPZndDNmk7zJFNJy5Qv85lZq8ou7Ka8d4DweHpmTaV5kqnOPOBkysxaVybphF7a7dYps2bSNMkUwy1TvpvPzFrUcCf0sjuhmzWVcZMpSddK2i5pXVXZlZI2SVqbPC6e3jB9mc/MWp+621F7zoN3mjWZWlqmrmP0GdU/HxGrkset9Q3raO6AbmatThKZhd2+o8+syYybTEXEnUDDRwd2y5SZzQbZhT2Udx8kylOao9nMZtBU+ky9V9IDyWXABWPtJOkKSWskrenr65v0yTzOlJnNBplFPVAKynv7Gx2KmdVossnUPwOnAauALcBnx9oxIq6KiNURsbq3t3eSpwPyWciIGChw6EfrKe89NPljmZml1OGR0N1vyqxpTCqZiohtEVGKiDLwFeC8+oZ1NEmoI0dp0352vvqr7HrzTdN9SjOzGZeZ1wX5LKUtexodipnVaFLJlKSlVau/B6wba9+6as/Sf91aAAr3b52RU5qZzSRlRP6kRRQ27CRK5UaHY2Y1yI23g6QbgAuAxZI2Ah8HLpC0CgjgKeCPpzHGUUXJnTPNrDXlVvRSeGI7xc27yZ+4qNHhmNk4xk2mIuKyUYqvmYZYxledPxX9H5uZtabcsvmoPUdxfZ+TKbMm0DwjoANEVTbl5m8za1HKZsidvJjC07uIoufpM0u75kqmqsZdCbdMmVkLy69YDMUSxY0NH+bPzMbRXMlU9WU+95kysxaWPX4+6sxTeHJHo0Mxs3E0VzJV9mU+Mzs2SWdWzRu6VtI+Se8fsc8FkvZW7fOxRsU7FmVE7pTFFJ/ZRRQ8WLFZmo3bAT1VqvtMuWHKzEYREY9SGVAYSVlgE3DzKLv+NCLeMJOxTVR+RS+FR7ZQfHoX+dOOa3Q4ZjaGpmqZ8lxVZjZBrwKeiIgNjQ5kMrLHzUVdbRSenPxUXGY2/ZoqmXJrlJlN0KXADWNsO1/S/ZL+TdLzZzKoWkkiv6KX4qbdxGCh0eGY2RiaK5lyy5SZ1UhSG/BG4F9H2XwfcHJEnA38I/CdMY5Rl4napyJ/ai+Ug8KGnQ05v5mNr7mSqXAyZWY1ex1wX0RsG7khIvZFxIFk+VYgL2nxKPvVZ6L2Kcgs6kFzOnypzyzFmiqZ6rz0rEaHYGbN4zLGuMQn6XhJSpbPo1IXprLpZ/hSX2nLHsoDQ40Ox8xG0VTJ1IJrL2l0CGbWBCR1A68Bvl1V9i5J70pW3wysk3Q/8AXg0oj0Nn3nV/RCQPEpjzlllkZNNTSC2poqXDNrkIg4CCwaUfblquUvAl+c6bgmK7Ogi8z8LgpP9tH2vGWNDsfMRmiqlikzs9no8KW+bfsoHxxsdDhmNkJTJ1PhUdDNbJbIraj0jy/4Up9Z6jR1MoUnOzazWSI7r4vMwm4K631Xn1naNF0ylTvzSDeIKJQaGImZ2czKn9pLecd+yvsHGh2KmVVpumRq8V3vPDJEglumzGwWyZ9SGeuq8KQv9ZmlSdMlU9nF3bS97EQAouBkysxmj8ycDrK9c3ypzyxlmi6ZAlAuCdstU2Y2y+RW9FLefZDSnv5Gh2JmiaZMpshnAQgnU2Y2y+RXLAbB0CObGx2KmSXGTaYkXStpu6R1VWULJd0u6bHkecH0hjnCcMuUO6Cb2SyT6Wonf+ZSCo9uobT7YKPDMTNqa5m6DrhoRNmHgB9FxBnAj5L1GaN8JWy3TJnZbNR+zsmQz3HonvWkeBYcs1lj3GQqIu4Edo0ovgS4Plm+HnhTneM6tsMtU06mzGz2yXTkaV91EqXNeyhuHFk9m9lMm2yfqSURsSVZ3gosGWtHSVdIWiNpTV9ffe5AUdJnyh3QzWy2anveUjLzOhm850nPBmHWYFPugJ7MtD5mO3NEXBURqyNidW9v71RPV5G0THnQTjObrZTJ0P7iUynvG2Do11vGf4GZTZvJJlPbJC0FSJ631y+k8Q33mXLLlJnNZrkTFpBdvoDBtRsoHyo0OhyzWWuyydQtwDuS5XcA361PODU63DLlZMrMZi9JdJy3AgolBn+1odHhmM1atQyNcAPwc+BMSRslXQ58GniNpMeAVyfrM8Z9pszMKrLzu8k/10MlmDVSbrwdIuKyMTa9qs6x1M59pszMDmtfdTKFJ/o4dM96ul57FpIaHZLZrNKUI6C7z5SZ2RGZjjzt5yRDJTzjoRLMZlpTJlPuM2Vm9mxtz02GSvilh0owm2lNmUx5omMzOxZJT0l6UNJaSWtG2S5JX5D0uKQHJJ3biDjrSZkM7eclQyV43j6zGTVun6lUGp7o2H2mzGxsr4yIHWNsex1wRvJ4CfDPyXNTy5+wkMLyBQze/zT5048j09HW6JDMZoWmbpna+4HbKG3a1+BozKwJXQJ8NSp+AcwfHjuv2bWfdyoUywz85FGi7NZ7s5nQlMkUSQf08qb97LvyJ42NxczSKIDbJN0r6YpRti8Hnqla35iUPct0TIc13bLzu+h42RmUtuzh0M8e90TIZjOgKZOpw32mgOzSngZGYmYp9fKIOJfK5bz3SHrFZA4yLdNhzYC205fQdvaJFB7bxtC6jY0Ox6zlNWcy1dM26rKZGUBEbEqetwM3A+eN2GUTcGLV+glJWctoP+dkcit6GVzzFIWnxuo6Zmb10JTJVGZ+J733Vlruo9/zUZnZEZK6Jc0ZXgZeC6wbsdstwNuTu/peCuyNiJaaLVgSnS9/Dtnj5jJw56OU+vY3OiSzltWUyRRA27nLoD1LDBQbHYqZpcsS4C5J9wP3AN+PiB9IepekdyX73AqsBx4HvgK8uzGhTi/lMnS+6nmoq43+Hz1E+cChRodk1pKac2iEhDrzxIBbpszsiIhYD5w9SvmXq5YDeM9MxtUomY42ul79fA5+fy39tz9E9+vPRm1NXfWbpU7TtkwBqDNHDBQpbTvQ6FDMzFIrO7+LrleupLx3gP47HiHKvsPPrJ6aPJnK03/1fWw9/h/ov3FklwgzMxuWWzafjt86ndLmPRz6hYdMMKunpk6mSut3H14euuvpBkZiZpZ+bc85nrYXnEDh0a0M3r3eLVRmddIyF87Vlm10CGZmqdf+olOgHAw9tInSvgG6Lniu+1CZTVFTt0w9S7uTKTOz8Uii47xTD1/yO/i9tZT3DTQ6LLOm1jLJlFumzMxq13bmUrp+5yziUIGj2Z2EAAAR6klEQVSD31tLceveRodk1rRaJpkauOkhT+ppZjYBuaXz6X7DKtSRp/+HDzL0m62NDsmsKTV1MpU/98gk78VHdtB/9X0NjMbMrPlk5nbS/fpVZI+fx6H/8xiHfumO6WYTNaVkStJTkh6UtFbSmnoFVaven11O7vlHJh8tbfF4U2ZmE6X2HF2vOYv8c5cytG4TAz9+mBj0gMhmtapHy9QrI2JVRKyuw7EmRO05jnvgT46su9+UmdmkKCM6zz+djpeeRnHjLg58+14KT2z3eFRmNWjqy3wAyhx5CzFUamAkZmbNr+15y+h+4zmop4OBOx+l/7Z1vtvPbBxTTaYCuE3SvZKuGG0HSVdIWiNpTV9f3xRPd2zlnf3Tenwzs9kgu7CH7tefTcdLTqPUt58D37mPwQee8U0+ZmOYajL18og4F3gd8B5Jrxi5Q0RcFRGrI2J1b2/v0Ueog+53vxiA8k7/92RmVg/KiLaVy+j5vReRW76AwXuf4uB3f0Vx+75Gh2aWOlNKpiJiU/K8HbgZOK8eQU3U/C+9nvzqZW6ZMjOrs0x3O12vWknnhSuJoSL937+fgZ8/TnlgqNGhmaXGpOcQkNQNZCJif7L8WuCv6xbZBGWX9lDc4EHnzMymQ/7kReSWzWPwvg0MPbKZwmPbaDuzMtdfpqu90eGZNdRUJmRaAtwsafg4/xIRP6hLVJOQPWkegz/1ZMdmZtNF+RwdLzmtMoTCA88w9Mhmhh7dQv6M42l/wQlkejoaHaJZQ0w6mYqI9cDZdYxlSrInzSP2HKK87xCZuf6FNjObLtl5XXT+9pm0rzqJwQc2UvjNVgq/2Ur+9CW0v/AEMnM6Gx2i2YxqmanCsyfPB6C0YS+ZFziZMjObbpk5nXS+7Azazz6RwQc3UnhsK4XHtpI/9TjanreMzOIekqsXZi2tdZKp5XMAKG3ZT/4FSxocjZnZ7JHp6aDz/NNpP/tEhtZtYujRLRSe2E5mfhf55xxP/rTjyHTkGx2m2bRp+kE7h2WX9AAw+O/rGxyJmTWSpBMl3SHpYUkPSXrfKPtcIGlvMhXWWkkfa0SsrSbT1U7Heacy5y0voeO3TodclsF71nPgxrvp/8mvKW7e7RHVrSW1TMtU5rhuAA78/c8oPrqThTe/5Vmjo5vZrFEE/jwi7pM0B7hX0u0R8fCI/X4aEW9oQHwtT2052s5cStuZSyntOkjhN1sZemI7xSf7UE8HbWcsIbdiMdl5XY0O1awuWiaZ0twjt+YeuuVRBu94io5XndrAiMysESJiC7AlWd4v6RFgOTAymbIZkF3YTfalp9G+egXFp3cw9JutDP5qA4O/2kBmfhe5kxaRP3kxmUXd7l9lTat1kqmRv4RuSjab9SSdApwD3D3K5vMl3Q9sBj4QEQ+N8vorgCsATjrppOkLdBZQLkP+1OPIn3oc5YODFDfspLBhB0MPPsPQA8+gnnbyJy8md/Iisr1zUcaJlTWPlkmmRoqBYqNDMLMGktQDfAt4f0SMnAPlPuDkiDgg6WLgO8AZI48REVcBVwGsXr3a/6HVSaa7nbaVy2hbuYzyoQLFp3dS3LCzMm7VQ5tQR57ssvnkls0nt3QBmR4PCmrp1lLJ1LKBj7K585MAxJ5DDY7GzBpFUp5KIvX1iPj2yO3VyVVE3CrpnyQtjogdMxmnQaYjT9tzjqftOccTQ0WKG3dTeGYnpS17KK7vq+wzt/NIcnX8fNTeUn+6rAW01DdSVbfelnd70mOz2UiVa/7XAI9ExOfG2Od4YFtEhKTzqNzZvHMGw7RRqC1H/tRe8qf2EhGUd/dT3LKH0ubdFB7fRuHXW0CQWdRD7ri5ZJNHptstV9ZYLZVMAWhhJ7FrgLJbpsxmq5cBbwMelLQ2KfsIcBJARHwZeDPwJ5KKwABwafie/VSRVOm8vrAbnr+cKJUp9e2vJFdb9jD06FZ4eHNl3+52ssfNIds7l9xxcyud2X03t82glkumlu38SzbP+ZSTKbNZKiLuAo7Zezkivgh8cWYisnpQNkPu+Hnkjp8H55xMlMqUdx2k2LeP0vZ9lLbvp/jkDgYBsiKzoJvswh6yiyqPzIIulMs2+m1Yi2q5ZAogM7+D8o7+RodhZmbTRNkM2d45ZHvnwMrlAJQPDlYSq779lHYdoPDUDgq/2Zq8ADLzuiqJ1cJusgu6yczvQl1tHpLBpqwlk6ncyl4K929rdBhmZjaDMt3tZFb0kl/RC0BEEAcGKe06QGnnAco7D1LcvId4YvuRF+WzZOd3kZnfVWnNSpadZNlEtGQy1faS5ez/5E8p7ztEZq4nPTYzm40koTkdZOZ0kD958eHy8qEC5T0HKe/up7Snn/KefopP7yQeq/onPJshM7eTzNwOMvO6qpY7UXveiZY9S0smUx0Xn8H+v7mTg1ffx5w/+61Gh2NmZimS6ciTOX4+HD//WeXlgSHKe/op7+2nvO8Q5b0DlTsKn9717IGg81kyczrI9HQkz+2VpK2n8lDefbNmm5ZMptpeeiLtv3Ma+//qx3Rffi6ZeW6dMjOzY8t0tpHpbIOlz06yolyuXC7cO0B53wDl/QPE/kHK+wYobtoNpfKz9ldHHnW3Vy47drcfXlZ3eyXx6mzzCO8tpiWTKYCe/3o+O3/4BHve/X3mfuICcqcvanRIZmbWhJTJoLmdZOZ2HrUtIohDBcr7DxEHDlHef4jygUHKB5Nka8seKJRGHBDU0Ya6K8mbutrIdLUnz5V1dbZVLic66WoKLZtM5V+0FICBf3mQQ7c+xvHP/FdPSWBmZnUlCQ23aB03d9R9YqhYSa4ODBIHByn3DxL9Q5T7hypl2/cRg6NMgaaklauzklxlDi/nUUe+st7RVtmnI49yHlurUVo2mcou7qbrD1cR/QUGbnqILQs+w7JDf4Wy/rKZmdnMUVuObFuO7ILuMfeJYpkYGKokWgOFyvLA0LOWi3v6iYEhKI8xvmw+eyTJas9VWrbGWm7PVablyWbcmb4OWjaZAljwP95EFEsc+uHjxN5BBn/wOB2vf06jwzIzM3sW5TKH7zw8loiAoRLlQwXi0BBxqPCsR3n4eaBA7OknDhWgWB77gFmhtiPJldqOPNOWrLc9u1xtWZTPQVvWiVhiSsmUpIuA/w5kgasj4tN1iaqOlMuydNtfsG3ll9j9zu+y8MY3037BikaHZWZmNmGSoD1Htj0H847uwzWaKJaJwQIxWKwkXUPJ8vBjqHh4e/ngILHrIDFUPLqv12jyWZTPHkm48tlKkpXPJeVZGF5O9iOXPbx++PVNftVo0smUpCzwJeA1wEbgl5JuiYiH6xVcvag9x6Kb30Lf+dew45XX0/byk1hw7SVkT194zKw6iiVisITas56GwMzMmpJyGZRrhwlOCB3lgMJwslWqJGFDpcp6ofJM1XIMlSgPFmD/IaJQKT9mq1i1jI4kV0myRS5b6Qf2rLJMJfnKVW0fTs6Gl3PJfrnMjF3GnErL1HnA4xGxHkDSN4BLgNQlUwD5Fx7Pksf+lB0XXs/QXU+z7Tn/iOa0kVncVdmhFJUvyWARBkvEoeKzr0vnKz+YKAXqaUMdIz66ET+ro354I3+WDd5e9/hs1sg9dzGLvvWWRodhZtNMGUF7HrXnYc7kjlFJyEqHk6sYKkFxeL1cSdYKpcMPCqVKQ0axDIVKclZJykoTS86q5TJHEqxshs5XrSQ7r2tyb2isU0zhtcuBZ6rWNwIvGbmTpCuAKwBOOumkKZxu6rLL5rLk139K4YGtDN71NMVHd1Le2Q8SyqrSNNmeQx051J6Fjsq14RgqEf0FKJQhI+LAUOULMWzkZPMj+wZOcPtRc9fX+fh1j89mldwp88ffycyM4YQs6exeBxEBxTJRLB1+juJwElauJF1Vz4f3K1WSsyiVp+VK07R3QI+Iq4CrAFavXp2KP8P5Fx5P/oXHNzoMMzMzmwBJh/tZpclUenxtAk6sWj8hKTMzMzObNaaSTP0SOEPSCkltwKXALfUJy8zMzKw5TPoyX0QUJb0X+CGVoRGujYiH6haZmZmZWROYUp+piLgVuLVOsZiZmZk1neYeJcvMbBSSLpL0qKTHJX1olO3tkm5Mtt8t6ZSZj9LMWoWTKTNrKVUDCr8OWAlcJmnliN0uB3ZHxOnA54HPzGyUZtZKnEyZWas5PKBwRAwBwwMKV7sEuD5Z/ibwKnmSMTObJCdTZtZqRhtQePlY+0REEdgLLBp5IElXSFojaU1fX980hWtmzW7aB+2sdu+99+6QtKHG3RcDO6YznhnQ7O/B8TdWq8R/cqMDmazqQYcl9U2g/oL0//zSHF+aYwPHN1Vpjm9kbDXVXzOaTEVEb637SloTEaunM57p1uzvwfE3luOftFoGFB7eZ6OkHDAP2Hmsg06k/oL0//zSHF+aYwPHN1Vpjm+ysfkyn5m1mloGFL4FeEey/GbgxxGeddLMJmdGW6bMzKbbWAMKS/prYE1E3AJcA3xN0uPALioJl5nZpKQ5mbqq0QHUQbO/B8ffWI5/kkYbUDgiPla1fAj4j9McRtp/fmmOL82xgeObqjTHN6nY5JZtMzMzs8lznykzMzOzKXAyZWZmZjYFqUymxptXKw0kXStpu6R1VWULJd0u6bHkeUFSLklfSN7PA5LObVzkh2M9UdIdkh6W9JCk9yXlTfEeJHVIukfS/Un8n0jKVyRzrT2ezL3WlpSnci42SVlJv5L0vWS92eJ/StKDktZKWpOUNcV3aLqkvf4a7WfW4HhqrktTFN+VkjYln+FaSRc3KLYJ1eMpii8tn9+E/o4cU0Sk6kHl7psngFOBNuB+YGWj4xolzlcA5wLrqsr+G/ChZPlDwGeS5YuBfwMEvBS4OwXxLwXOTZbnAL+hMo9ZU7yHJI6eZDkP3J3EdRNwaVL+ZeBPkuV3A19Oli8Fbmz0zyCJ5c+AfwG+l6w3W/xPAYtHlDXFd2iaPo/U11+j/cwaHE/NdWmK4rsS+EAKPrsJ1eMpii8tn9+E/o4c65HGlqla5tVquIi4k8ot1dWq5/u6HnhTVflXo+IXwHxJS2cm0tFFxJaIuC9Z3g88QmWKjaZ4D0kcB5LVfPII4EIqc63B0fGnai42SScArweuTtZFE8V/DE3xHZomTVF/pckE69IZN0Z8qTCJejwt8aXCJP6OjCmNyVQt82ql1ZKI2JIsbwWWJMupfk/JJaNzqGTlTfMekktka4HtwO1UWgT2RGWuNXh2jDXNxTbD/l/gg0A5WV9Ec8UPlYrnNkn3SroiKWua79A0aIb3ONrPLG3G+g6lyXuTy9XXNvIy5LAa6/GGGREfpOTzm+DfkTGlMZlqCVFpH0z9uBOSeoBvAe+PiH3V29L+HiKiFBGrqEwXch7w3AaHVDNJbwC2R8S9jY5lil4eEecCrwPeI+kV1RvT/h2apY75M0ublH6H/hk4DVgFbAE+28hg0l6PjxJfaj6/ev0dSWMyVcu8Wmm1bfiyRfK8PSlP5XuSlKfyBf96RHw7KW6q9wAQEXuAO4DzqVw6Gh6MtjrGw/GrxrnYptnLgDdKeorKpaALgf9O88QPQERsSp63AzdTqYya7jtUR6l/j2P8zNJmrO9QKkTEtuSPcBn4Cg38DCdYj6civjR9fsNq/DsypjQmU7XMq5VW1fN9vQP4blX525O7mV4K7K1qgm2IpL/NNcAjEfG5qk1N8R4k9Uqanyx3Aq+hcj3+DipzrcHR8admLraI+HBEnBARp1D5jv84It5Kk8QPIKlb0pzhZeC1wDqa5Ds0TVJdfx3jZ5Y2Y32HUmFEX7/fo0Gf4STq8Rk1Vnwp+vwm+ndkbPXqFV/PB5W7fn5D5drlRxsdzxgx3kClebJA5Zrq5VT6sPwIeAz4d2BhHLlj4EvJ+3kQWJ2C+F9Open3AWBt8ri4Wd4D8ELgV0n864CPJeWnAvcAjwP/CrQn5R3J+uPJ9lMb/TOoei8XcORuvqaJP4n1/uTx0PDvarN8h6bxc0lt/TXWz6zBMdVcl6Yovq8l3+EHqCQuSxsU24Tq8RTFl5bPb0J/R4718HQyZmZmZlOQxst8ZmZmZk3DyZSZmZnZFDiZMjMzM5sCJ1NmZmZmU+BkyszMzGwKnEyZmZmZTYGTKTMzM7Mp+P8Bmc+mDLj80QMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 固定随机种子\n",
    "paddle.seed(0)\n",
    "# 定义网络结构\n",
    "model = Linear(2)\n",
    "# 定义优化器\n",
    "opt = Adam(init_lr=0.1, model=model, beta1=0.9, beta2=0.99, epsilon=1e-7)\n",
    "train_and_plot(opt, 'opti-loss5.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**动手练习7.5**\n",
    "\n",
    "学习AdamW算法，通过飞桨 API调用LeNet网络和MNIST数据集，使用paddle.optimizer.AdamW作为优化器训练网络，进行简单的拟合实验。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 7.3.4 不同优化器的3D可视化对比\n",
    "#### 7.3.4.1 构建一个三维空间中的被优化函数\n",
    "定义OptimizedFunction3D算子，表示被优化函数$f(\\bm x) = \\bm x[0]^2 + \\bm x[1]^2 + \\bm x[1]^3 + \\bm x[0]*\\bm x[1]$，其中$\\bm x[0]$, $\\bm x[1]$代表两个参数。该函数在(0,0)处存在鞍点，即一个既不是极大值点也不是极小值点的临界点。希望训练过程中，优化算法可以使参数离开鞍点，向模型最优解收敛。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class OptimizedFunction3D(Op):\n",
    "    def __init__(self):\n",
    "        super(OptimizedFunction3D, self).__init__()\n",
    "        self.params = {'x': 0}\n",
    "        self.grads = {'x': 0}\n",
    "\n",
    "    def forward(self, x):\n",
    "        self.params['x'] = x\n",
    "        return x[0] ** 2 + x[1] ** 2 + x[1] ** 3 + x[0]*x[1]\n",
    "\n",
    "    def backward(self):\n",
    "        x = self.params['x']\n",
    "        gradient1 = 2 * x[0] + x[1]\n",
    "        gradient2 = 2 * x[1] + 3 * x[1] ** 2 + x[0]\n",
    "        self.grads['x'] = paddle.concat([gradient1, gradient2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "对于相同的被优化函数，分别使用不同的优化器进行参数更新，并保存不同优化器下参数更新的值，用于可视化。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 构建5个模型，分别配备不同的优化器\n",
    "model1 = OptimizedFunction3D()\n",
    "opt_gd = SimpleBatchGD(init_lr=0.01, model=model1)\n",
    "\n",
    "model2 = OptimizedFunction3D()\n",
    "opt_adagrad = Adagrad(init_lr=0.5, model=model2, epsilon=1e-7)\n",
    "\n",
    "model3 = OptimizedFunction3D()\n",
    "opt_rmsprop = RMSprop(init_lr=0.1, model=model3, beta=0.9, epsilon=1e-7)\n",
    "\n",
    "model4 = OptimizedFunction3D()\n",
    "opt_momentum = Momentum(init_lr=0.01, model=model4, rho=0.9)\n",
    "\n",
    "model5 = OptimizedFunction3D()\n",
    "opt_adam = Adam(init_lr=0.1, model=model5, beta1=0.9, beta2=0.99, epsilon=1e-7)\n",
    "\n",
    "models = [model1, model2, model3, model4, model5]\n",
    "opts = [opt_gd, opt_adagrad, opt_rmsprop, opt_momentum, opt_adam]\n",
    "\n",
    "x_all_opts = []\n",
    "z_all_opts = []\n",
    "x_init = paddle.to_tensor([2, 3], dtype='float32')\n",
    "# 使用不同优化器训练\n",
    "for model, opt in zip(models, opts):\n",
    "    x_one_opt, z_one_opt = train_f(model, opt, x_init, 150)\n",
    "    # 保存参数值\n",
    "    x_all_opts.append(x_one_opt.numpy())\n",
    "    z_all_opts.append(np.squeeze(z_one_opt))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义Visualization3D函数，用于可视化三维的参数更新轨迹。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from matplotlib import animation\n",
    "from itertools import zip_longest\n",
    "\n",
    "class Visualization3D(animation.FuncAnimation):\n",
    "    \"\"\"\n",
    "    绘制动态图像，可视化参数更新轨迹\n",
    "    \"\"\"\n",
    "    def __init__(self, *xy_values, z_values, labels=[], colors=[], fig, ax, interval=60, blit=True, **kwargs):\n",
    "        \"\"\"\n",
    "        初始化3d可视化类\n",
    "        输入：\n",
    "            xy_values：三维中x,y维度的值\n",
    "            z_values：三维中z维度的值\n",
    "            labels：每个参数更新轨迹的标签\n",
    "            colors：每个轨迹的颜色\n",
    "            interval：帧之间的延迟（以毫秒为单位）\n",
    "            blit：是否优化绘图\n",
    "        \"\"\"\n",
    "        self.fig = fig\n",
    "        self.ax = ax\n",
    "        self.xy_values = xy_values\n",
    "        self.z_values = z_values\n",
    "        frames = max(xy_value.shape[0] for xy_value in xy_values)\n",
    "        self.lines = [ax.plot([], [], [], label=label, color=color, lw=2)[0]\n",
    "                      for _, label, color in zip_longest(xy_values, labels, colors)]\n",
    "        super(Visualization3D, self).__init__(fig, self.animate, init_func=self.init_animation, frames=frames, interval=interval, blit=blit, **kwargs)\n",
    "\n",
    "    def init_animation(self):\n",
    "        # 数值初始化\n",
    "        for line in self.lines:\n",
    "            line.set_data([], [])\n",
    "            line.set_3d_properties([])\n",
    "        return self.lines\n",
    "\n",
    "    def animate(self, i):\n",
    "        # 将x,y,z三个数据传入，绘制三维图像\n",
    "        for line, xy_value, z_value in zip(self.lines, self.xy_values, self.z_values):\n",
    "            line.set_data(xy_value[:i, 0], xy_value[:i, 1])\n",
    "            line.set_3d_properties(z_value[:i])\n",
    "        return self.lines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "绘制出被优化函数的三维图像。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. \n",
      "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
      "  if data.dtype == np.object:\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:101: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead\n",
      "  ret = np.asscalar(ex)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAADuCAYAAAAOR30qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsvWmMJOd1rvl8X6y5Z+29kM2lm1tTpEhRCzUWNNI1NBgNDHtkyTIBAxYw8hg2jIFsA4YI2AZs/9BiWBprML/mwjMWdO0ZCzLG9siG7vha1sVciRQpSqQlUpS6urt6rb0qK/fMWL75ERlZGUstWVXdqm7FCzTJDmYsGRnxxolz3vMeoZQiQ4YMGTL85CF/0geQIUOGDBkCZIScIUOGDMcEGSFnyJAhwzFBRsgZMmTIcEyQEXKGDBkyHBNkhJwhQ4YMxwQZIWfIkCHDMUFGyBkyZMhwTJARcoYMGTIcE+hjfj5r68uQIUOG8SH286EsQs6QIUOGY4KMkDNkyJDhmCAj5AwZMmQ4JsgIOUOGDBmOCTJCzpAhQ4ZjgoyQM2TIkOGYICPkDBkyZDgmyAg5Q4YMGY4JMkLOkCFDhmOCjJAzZMiQ4ZggI+QMGTJkOCbICDlDhgwZjgnGNRfKkGFXKKXwPA8ATdMQYl+eKhkyZCAj5AxHBN/38TwP13Xp9XrD5UIINE0b/pFSIqVECJGRdYYMMWSEnOFQ8H0f13WHUbEQYki4SgVurSFRx9frdDpMTEyg63pG1BkykBFyhgNAKYVSCsdx+PGPf8w999xDLpeLkDAwJNY0gu33+ywsLFAoFOj3+5F1pJRompYRdYafOmSEnGHfUEoNI2Lf9wFot9sopcYmy/Dzmqal7sPzvAhRh5+Npz8yos5wNyEj5Ax7Ik7EIQmGf0aj4v1ip/V2IthRoo4/AEYJOp6nzpDhTkJGyBl2RKiYcF13SIJxwpRSDqPlcTAukY9L1CsrK8zMzGCaZmpBMUOG44iMkDMkkEbEUqZL1o86Qj7IdtIIdnFxkampKRzHod/vJx4iOyk/MmT4SSIj5AxDKKWGiom9iDhEGrEeF2ILiXcU4bG6rovjOKmfz4g6w08KGSFnQClFp9MBtiPOvYg4hJTyJxohH2S/o/8OsRtRh1rqUdVH1vSS4VYgI+SfYow2c7z88su8/e1vT0SUe0EIcVtyyLcaexH1aApHKcWNGzc4c+ZM1vSS4UiREfJPIXZq5jiqSHe/MrjjRMg7IY2oHcdhY2OD++67L7XpJTyfuq5nRJ1hLGSE/FOC0WaOMKIdJYjbpZYYXe9ORpriBLYfMr7vR1rIw3WyppcMuyEj5Lscac0caQRwUELeab29ouTjlrIYB7t9t926E7Omlwx7ISPkuxS7NXOk4agi5MM0i9wpOEhnIuxfS+37PteuXeP+++9PzVFnyo+7Fxkh32XYTzNHGm63nvhOJpSDEvJOiP8+juNQr9eHD8m07sRMond3IiPkuwRKKXq9Hs1mk0KhMJZ0DQ6XsogTcq/XG+ZJ71bcSuLzfX/XlMVoPSBrerm7cPfeMT8lGG3mqNfrXLlyhSeffHLs7RwmZRGuV6vVuHjx4rBw6Ps+tm1TKBQoFAoUi0Xy+fxYD4rjiFudjgkJeSfsRtSQNb3cycgI+Q5FSMSh5EoIgWEYByJVOBwhN5tNXn75ZXRd56GHHsK27SE5dLtdWq0WrVaLjY0N2u02vu+Ty+Xo9XosLy9TKBTuKKI+6pTFUW1/nKaXhYUF7r33XgzD2FGil+H2IyPkOwyjzRyQlK6F2uJxMa4OWSnF+vo6CwsL6LrOE088QalUAhgqCIQQ5HI5crkc09PTkXU7nQ6vvfYanU6HtbW1oY1nLpcbRtOFQoFcLnfsiPq4EvJOSCPqWq3GfffdB5CoOSildo2oM7K+dcgI+Q5BWjPHUUnXwu3tZ12lFKurq1y+fJl8Ps8999yDlHJIxvvdVz6fR9d17r///si2O50OrVaLZrPJysrKsKU7JOrwz3Ek6qPCXimLo9pH2P69V3fiKLKml1uLjJCPMfZq5ojjMIS817pKKZaXl7l8+TLlcpknnniCfD7PzZs3Ew0QB0VI1Pl8npmZmeHycNxTmPoYJep8Pp8g6ltNDLcjQr4dD5txtdRZ08utR0bIxxD7beaI41YQslKKxcVFFhYWmJiY4Omnn8a27eH/38nt7SgLX1LKIeGOYpSoG40GS0tLdDodhBB0u12uXr1KqVSiWCxi2/aREcOtJuRQN37cME7Ti+u6LC4ucu+992ZNL2MgI+RjhPCiXltbo1wu71tDHOKgfhRp6/q+z82bN7l69SqTk5M888wzWJZ1pPs8LEaJenZ2drjc931eeeUVbNuOELWUMhFRH4Sof1oJeTfEr9N+v0+j0YhoqUOE5y9rekkiI+RjgHgzxxtvvMG73/3u23phhjlk3/e5fv06165dY2Zmhre//e2YprnnescJ4avz7OxsRAvteR7tdptWq8XW1hY3b96k2+0OiTosJBYKBSzL+okRw+1KWdxKeJ63Y44adh/J9dMs0csI+SeI3SZz/CQuvtXVVa5cucLc3BzvfOc7MQxjz3XupDZpTdMolUqJAuQoUW9ubnL9+nV6vV4kAh8l6jtNZfGTQEjIOyFreklHRsg/ARxkMsetguu6XLt2jStXrlAqlXjXu941VofdnUTIO2E3og4LiaNEHX7fa9euDYnaNM0jI4ZbrbK4Hb+X67pje2vDeE0vjuOwtrbG6dOn7xqizgj5NiKtmWOnG+9WR0mO43D16lWWlpY4ffo0Dz30EP1+f+x258MUEo87NE2jXC5TLpcjy9fW1lhaWkLTNNbX17l69Sr9fh9N0yLRdLFYxDCMsX/HW51Dvp2yuqNCWkGx0+mwtbXF6dOnd+xO/MIXvsDzzz+/r7e944CMkG8DdmvmSIOmaUd+QYfo9/tcuXKFlZUV7r33Xp599lk0TWNpaemumPxxO6BpGpZlcerUqchy13WHEfX6+jpXrlzBcRx0XU+kPnbLy9/qHPLtIOSDRsjjwPM8dF3fVUv9t3/7t/z+7//+LT2Oo0RGyLcQ+2nmSEPYcXfQCzotuu71eiwsLLC+vs6ZM2d497vfHbkpb7dB/Z2Mnd5edF2nUqlQqVQiyx3HGRL16uoqCwsLQ6IeLSQWCgUMw7jlb0eHubaO0z5c193xjW70uryTUhcZIR8xwupxq9UaviaNq7nUNO3ALdDx6Lrb7XL58mU2Nze5//77eeihh1Kjo6N0e7uTboCDYFzCNAyDarVKtVqNLB8l6pWVFZrN5lB7Hqo8Ron6qHA7ImTP81JlkkeJ3Qh5FHfS9ZgR8hFhtJnDdV1eeeWVA0vXDkPIYXTd6/W4fPky9XqdBx54gEcffXTXYzkKtzeAra0tLl68iFJq2+HNtMkVC2j6rY2YbheO6o1gJ6JeWFgYKm+Wl5dptVq4rothGImI+iAWp7eLkA9jv+q3e7hX1jHOziLM9O3sRci3I21y1MgI+ZBIm8wRXgQHfTKHUe5B4Ps+b7zxBt1ulwcffJDz58/v6zgOmnoI19va2mJ+fh6A+weTLsLor/6di5jLLToTJupkCXtugmKxONQ934ma21sZdQkhKJVKzM3NDZeFUrBms0mr1WJxcZFWq4XneZimmbA43Y2oblV9YhSe5439u/rtPu7VdbzNFu7FVUTBwnzs1I6fd1131yh8a2srkT467sgI+YA46GSO/eAgEXKz2eTixYs0m01Onz7N6dOnxzqWg0bIzWaTjY0NPM/j3LlzVCqV4VtC6Emx8r0lHFdRXO3Bag8KdZoniiivw3e+852h3nc0+jtKGdlR4yehQxZCYJomk5OTTE5ORj7b7/eHD7+bN28OidqyrEQxMby2jkuE7DU7eFc3cK5t4K81QIGcCFrk9ZO7k6nruol2+lHU6/XE28dxR0bIY2K3Zo6jwjg2mvV6nYsXL+K6LmfPnkUIQaVSGZswxiXk0Ize930KhQJve9vbUj/ntXs4a43IMtFxMBe2OOP5aBUX+74Z/FyRrk5ERhaqE0aJ+jjIl25H6/R+rykhBJZlYVlWKlGHEfWNGzdotVqRbS8uLg4j6qOOmHcq6iml8DdauNc3cK9tgCbw11vbH5ACfyswjtJO7k6me5F+rVbLIuS7FbezmWM/EXJIiAAPPvggExMTANy8efNIp0en7Xd+fh4pJefOncOyLF5//fUdP9+9tpZYpk8WcdeaAHhbbVr/dgVtooB0PWbPTGPfdy/miQlc3xtacY7mUk3TTORS77Rc4W44CsIfJeqpqanItm/evEmtVsNxHK5duzYcGjA63SUcGnDQ8zpKyKrv4i5t4a02cC+voroDvbAAtOg9JKt5/I02CNBPlNkNe+WQs5TFXYhxmjlGERa7DkLauxHyxsYGly5dQtO0YYpgFLdKvra5ucnFixeRUvLwww8PmyVGO9fS4Gw00KsF3Np2FCRTbiLNNugvtWi9cZ3WG9cRhkb+4VOYkyVO3jONds89QPQVvdlsRiI/27aHRO153i1rWDlOEfK4CGscpVKJM2fODJcrpXad7hIn6t2OTymF1uij3lymvdLAW22AUmBq0N++rmUlj19rR49vUPiVEwWEtfvb0H4IOUtZ3CUYt5kjjsPk6uJFPaUUGxsbXLx4EdM0eeSRR3Y0hD9oQXAnIg+JWNO0CBGPrrcTISvPp/nmDZTjoZVy6EUbv9HFa3YTn/W70S4r5Xj0F2u0X78OgD5RwDo9iXXPFMZcJTWX2u12h6/o3W6XV199deixPFr0OqwV553uZZFG+ELsPN1l9LymTXcp5PPkPQ2j7uCv1HGX65zyXFzvxvb2yzaqHvvd4+oJAUoI5GwJfY90BWSE/FOBUDFx9epVbNtmenr6UNK1g+Q8wxyyUoq1tTUuXbpELpfj/PnzFIvFfa17kH2OEvJeRBwize0tPF/dmxsoJzgWr9HBa3TQijaarmGcqNJd30JzFMI2IhE0gDA03M3tZe5mC6/Rof3DgKCN6TLmyQnMk1WM2QrS0IeEMjMzw8bGBk8++SRSSjqdDs1mM2HFGW9z3m8h8U6OkMfd/ihRh0MDlO/jrTfp3tjAuVCDxiqq69EfrOOYAjN2CUrbxIsTcs8BKZDVPOgafsfBX94CQHvqDHthr+aTWq3Gvffeu6/veVyQETLpkzlCo+3DSNfi42/2Cykl6+vr3Lx5k1KpNJzOsd91DxMhb25uMj8/j67ru0biIXZLdXSvrCaW6UWb/lJte/1qAbOSx9lo4re2p1Do1SLuaj267kQRdy1Y5qxs4axs0btZxl1rok8WMecqGLNljLntqGgnc/u4cdC1a9cihcTRYuLtLiTeDsIfRyPst3p4603ctQaq1sZd2gIvuMb0oonqRtnXLOVhPfqAbW3VCccaKF0iShZK00HKIGcMyJkifgMwNLTpvUeC7XWeMpXFHYbdJnPoun5gQoWgjXbcSFUpxdLS0pAQn376aXK53FjbOCghb21t0Wg0WFhY4NFHH933jLzdCLmTQsh+f/ucCsCrtfAsHa/ZRa/k0fIWXqePkMkbTWgpy6QEpXDXG7jrDXgDjNkyJzYb1Nd+gDldRp8uYUyW0Irbk052Mg7aqXtuVOvb6XRuKUnf6gh5NwWEavXwNlp4G038ehd3ZQvVCdJJIm9Cux9ZRxRsVCtm6qMgcgVqgpxpoYoaXt9BNvp0uz3sTjRidvoOGqCdqKT+/uOiXq9nRb07AaF0LSz8pGmIdV0/1Ky4cbTEvu+zuLjIlStXmJyc5KGHHqJer49NxuF+xyHk0YjYsiyefvrpsfa3EyH3V7bwWtEbTlh6JA0x2MBwmbvVxt0aFHkmwJirgK9way2U4+E1krlnr96JbQ+8ehvNVTjXN3Cub4AQCE0iNA19soA+UUSfLAYPgGoeOVI8Suueizdl1Go1er0eS0tLQ2VCGE3vVfDaD25LSqTn4S7W8Gpt/K120JSxUodBignbgFheXytaeDFCxov99gL8rQ6eBGOyGFwfUuAvB2824WOgUCzgdZrD1RQK0Qzut+v9Glsvv5zQUI/m/vfTxJTlkI85xmnmuB0Rsu/73Lhxg6tXr0amc2xubrK5uXmg/e43hxwWCQ3DGEbE3/rWt8be307E0bm8klhmVIs4y7XIMm0ij78ZrbTr1QJerY1HO9wJ5okKQoAwdbytNiiFVs4l8pJaJY+/FdveRAFvo4VyfZzFGs5iDX2mPEyJyJyJVs5hzFWRpoYs5YJcd9FG5sxEU0aotDl58uRQmdBsNocFL9iekh0S9TjDV48iQlaej9/q4bd7+PUufrOL3+ji1ztUez30zhLDR2Ma+ZZsvHih1Y+Tr8APH4iaRFZyCFtHNXvIehd/IG2Us8m3Lb8RDXZkJY8a/G7n/qunIWfQbrdpNpup013y+fxwpuJORdqtra2hHPROwU8FIR+kmeMgKYdR7JZD9jyP69evc/369dTpHIf1soj7wo4ijYhvBToLSUImJXIXRvISlDkzIN0Qg2iov1gbrKOhV4vIvAVCBlHy4DNazkwQsjR1Emdz5Ab2O338Th/lBcWqUWjVPPgKWbCQORNZsNB6HYSl4ygbwzKoWgUmS1WEqQ2LnGnDV+OjonYqJO4UISvXQ/VdVM/F77uoroPq9vG7Dn6nj0DgrTXw2/2h1lcWLfzmNvmJnIEey/mmkS8pAag/+jZiaoE0DREcT72Nv9FCmy2j6l1Gj161YhOqSzYqRsjC0lGAnMgjC0E7dLFYTBSxw+kutVoN3/e5cOHCkKhHm4c6nQ61Wi0j5OOEg2qI4XBFuXD9OKmG0zlu3LjBqVOndpzOcVhCTktZbGxsMD8/j2mat5SIAXprW/jt2A2nS5yNKNkpwE9JQ/idfmKZ6m0ThnI8nNU6+lQJd6uF0CVapYg0NJACYWhDdQeQfM0mUH1EIAVeXBNrm8NlYWpEnyhgDJbVf7AcLJut4K1sgQh0tMLQ0WfKmM0uU5pkWpOg5RG2jrvZx/XW6HtLrLgunu+jIdA1HV1qaLbBzGaT7tXX6foKHA9ZyeEub8FIhKrNlvFWokVPWbCj5GebETIGkKUcfjfaOZmGCPkKkJNF5CCH77f7qGYveADEjkHFf7uCiWpFl8mciRcj5PABop/enUBDDbWUkq2tLd7ylrcA0SLtpUuX+JM/+RMuXLjABz/4Qc6fP88HPvABPvrRj+75vePwPI+3v/3tnD59mq9+9atcvnyZ5557jvX1dZ555hm+9KUv7eptPS7uSkJWStFsNun1ehQKhQN5TBw2ZTFK6PHpHM8+++yuVe6jsN8Msb6+PtQv70c2d1j4vs/Sd3+Mp3zkZB7pKfytDsZkCWdlK/rZnI7sxG9WKxodM8g9x2RxmNuyOOX6uOsNhKUPGw8cQ5CrFhGaBj0XdAlucF5kKYcfI+QwrTEKrWzjdmNFrLRmhfB8q+BhoRwPd72JP6q31iQCBb4KCldArprHr4ffdXDcJZNc28dvbx+Lo4MWSxfEH2QibyUiUa1s48WPPyX3GiFfKRCTeaSQwVtE38VrdJC6xFuOkW8sqha2gYodlyxY+DFCVv3YtW3rqPCBtwchh4i3TY8WaU+ePMnXvvY13vve9/K1r32NN998k34/+VDeD77whS/w2GOPUa8H3/2Tn/wkv/M7v8Nzzz3Hb/zGb/AXf/EX/OZv/uaBtp2GO89maxf4vo/jOHS7XWq1Gjdv3jzwbK2jyCH3+30uXLjASy+9hGEYPPvss9x///17So4O4/YW5pDX19d56aWXuH79OufPn+epp566pWQcTqt+4Vvfwru+ifAUar2FV2vjGLDVbeGb0cq+MpKXn15NyvuMaiHxCm1MFIdpiuGyyuBzCoy+wl1pBEXBjSbK9RAFC22mhD6RR58uIYsW4bt1Wuokvn0Av5u8seMpEpEzo2TMdupjFDKF3HNm1L1MAbIdJT7HEIlIVBZTXM/iOV+iRC4KFnK2jKzk0aZLiJId5Oalhr/awF+p49fawW/Zjd0Llh4lckCWbBJwY9exoQ29KkJog/WEpSP3IXeD/Xkhh65573jHO/iZn/mZfW13FNevX+cf//Ef+bVf+zUgCPS+/vWv85GPfASAj33sY/zd3/3d2NvdDXdFhJw2mcMwjEPlgA9DyL1ej8XFRTY3N3n44YcT0zn2wkEjZKUUjUaDxcVFer3ebYmIw1bmF198kenpaZ649yFWvvdS5DOGkhgtL9Culiw8TSJafegkc91+LyX/nUIs6QeTsszbjl79ZlDcYrK4rfaQAjmQw+lzFVAK5fp4nV5CwSEsPUkmKUVJrWTjxr6b0JO/f+K7CpF4O9DKyWjerBZQq9H0T6veJK7J8ZtdRMFCWHrQkqxL6Lv0fRfDA9XqIotWMvLtx657QxuJ5LePy1uLHkPiASZFMp9fyQ+LfcPVfAWmjv7AzL6Dp70I+SjmEv72b/82f/qnf0qjEaR41tfXqVarw/3ec8893LhxY7dNjI07mpBDMohriCGQL+1W3NoLB/kxR6dzTE9PY1kW9ww8GMbd9zgR8mhrtZSSarXKW9/61rH3Ow5CzfTly5fxfX+oEFn5xr8lPmtOlegP1BWq0UMSqCEcx0FWc6hGNygM6YEELnLmJYnc86hULrIsntaQKZ8zNNzRXLGvwPdwlqLqD30iUHqInIm0DYQukbaJcly6nW6Q90UicyZ+vbtN/KQG1qhYTh1dJshKqybJXeaMBCFLBH7BCvLVukRJQVHT8F0Xz/Hw+w6uUFhtB8F2UU1N5pCbHUbjctWLk2/yuAIS3SPvLEj4UgRGQa0dVgjeJETZCqR2jos+s7uZ0Cj2IuR6vX6oOslXv/pVZmdneeaZZ/jGN75x4O2MizuakEcn0cYJ9LAph3HQ6XS4dOlSZDpHo9HgypUrB9refh8GIRHPz89j2zbnz58H4NKlSwfab7jN3fYfTrG4fPky1WqVZ555hldeeQXTNFGeR/PHN5PreMmHi56zA71qO7jR9ckCVimPt9mKeF34RQutHiUzv2gim9FXdn2ymFRITBTwY8v0iQLuapRctGIOt5WeK1adPt4gPRBK5cLEiwvoQgTfb1BMxNDA9dAm8iAlQoDSNYRSaPkwrSAQpkQ5fhDViyC3K0wDoes0mg2K+UIQrQuBLNko1w8KlZ4X5Lo9f/hCoFXzeIOIWQ7+WHNlvHY08vV7TiRH6UuB2mpHHoBaSgRL/HIQSf23rBZQ8YefFn8rEEFKZLaM6vZRjS4ib+BvthFSop3av2Y4nKCyE2q12qE0yN/85jf5h3/4B/7pn/6JbrdLvV7nE5/4BLVabfgwuH79OqdPnz7wPtJwRxMy7KwqOGyEvB+EFd1Wq5WYznGYwtxeUEoNi3W5XI7HH398mJpot9sH3m/Y5JEquVKK1dVVLl68SKVS4emnn8a2oznD5qXlxGu4sA2c9WR05cZegd2NFkJK3EYHbaADVn0XaWg4MUKWpg5ECbTr9IjfnlKTpAjtEktwk+fLT1NmxHPFtr69zFeonht0GsaISZ+r4MbSAvpcikqilMNvdMkBXqcZpDk8FQm55VQhEXUGD48oQcZzvsLQ0DvRZbJsQy2a626124nUh4qTbyWfSN0IQyYyRqrZRdgGomQj1CC9v96Mfm5A2tpcOXig7ROu6+7aOHXYppBPf/rTfPrTnwbgG9/4Bn/2Z3/GX/3VX/FLv/RLfOUrX+G5557ji1/8Ir/wC79w4H2k4Y4n5J1w0BbiONJE+uF0jnBMUpoB0a0g5DgRv+Utb0l4NBzme4frjn7f0ODo4sWLFItFnnrqqR1vhMYPryWWmRNF+kvRJhd9opggLWEbuIPURGhEhBRI20SfKSOkxGt28Ns9ZNtJEK3Ri9KBAnobDSK3eEqOVmgykdYQOTORJtAmColj1soFvJjfhjT1xLGpeGELEgUxYRsJ5YSsJqNVkTKXUMVz8WYy5ysrefzYg1EzDDyi+8x5ktHG564Fdiyt4espD+zwDUOXuLaOMiWWI4Lz2HOD4uRsOUnag++s3zPJONjLnP5WeSF/9rOf5bnnnuMP/uAPePrpp/n4xz9+pNu/awn5KFpPw+aQkKBGp3M8+OCDw66tndY9qpTJfog4xFEQcrjPMB2Sz+d58skndzU46jfaOM0O1skJ+qt11CDq9DvJ9nPNMhKNGma1OMwzhzCmSrirDZyRaNU4EUxD0UoKt9FBdZ1hZ98oHFtidaPnoW8LzFjkKCt5/FiOWi/ncGOSrrTmEpFSRUxE1in6Zlm2E/IwrZzDjemDRUohONFkYRsJctcq+cCDOLKtxKaSD4BKDhWLfAuVZCTvNLrDtxFfglvQ0XWBJm1o9jCaDt6EFUTII/dH4tjLuWH0rd0zXgPHfnLIR9U2/b73vY/3ve99QDAM4qWXXtp9hUPgjifkW9nzH5Jqu90eTlIenc6xG44iQvZ9f1is24uIR/d7WEIezUvvZ59CCOqvX6G3FuiMpaFhzlVAiESrNFIki3SA30+ml0QKiwhEpAAnizZaOYc0DbxWd+gYp1KMiAqVMk43qoXu9rrEZf1uitIj0VwiUhpJUiPrlNx23sKNkWGiLZmUKLpgRhzxYKB22KvRQyS3Jcu5oBg5+jHbSBCyisv8yjlsywjSwV0HGl0M3UDbjG7La/eibyeFpE5a2Aaq3kFODboux8DdOC0E7gJC3guH8QXwPI/XXnsN0zQ5e/bsWD/wYQ3Qfd/npZdeolAo7IsUQxzUDxmCi/y1117DsqyxJHMC2Hrj6vDvvuPRXdrEPjmJKNgYpRxevY3X7mFOl3FiEZdWtJNpA00MUxijO/K2op/zm11cGBKVsA20Uo5ut4M2WcRvdocyroQREWA5UbWc0gTUO5FMs2dqEO/sSzFc18s53Jg+eN9yt3gUnRKtyqKdfDCkEHk8LTPqEzHcpW1C3J84nvooWWDoaHPloKjY6gWqkpHfTwC6EpE0jWsIzH70uFp+j/gVHGq19XvHS1fA3oRcq9UiU7vvFNzxhLwb8RmGMbRO3C9GJWTdbpezZ88eeSV1t32HhvSO4/CWt7xl7NeugzwItra2mJ+fp91u8/jjjzM7OzvW+ubs1M79AAAgAElEQVSWk5gCIqSgv95ADTq9IEhBSNsMfAtG8pJ6KUc/Fj0ZU0ni1ieLePH0QrUQISDVdVCWgd1wcRkoDwomWqWAUIEnhd/u4bf76JPJ7jxjsoQXk3hpJQvWo4TWc/vE2yBG27VDpGuZY3K3yQJezD9Y2gZejJATxcc0Ip8o4McebtJKplsika8UiLKNGCFfv9VDy1v4K41oYsaJFQtTtNmuJdFjx1o0c6jO9uccS2IMHi4/aixh/bgVMWPaq+nDdd1dzenr9ToPP/zwrts4jrjjCXk3hCmH/RBy2nSOmzdvHmmf+l77vnjxIoVCgSeeeIIf/ehHWNZ4r3Hjol6vMz8/j1KKhx56iIWFhQNZftpLSa2pOVOhvxxND3jt3jASNqdK9Nw+WstJ+kpAauQnjSSxSNtMRoQ5M0JmfquPXspH0ye6ROYsxKyGkBLlevg9J9WHV/NFYr+WF/2cL8DdbEYi6yBHmuzYi+dkRZo3cTxaNTS8eAPKZCE6sRkQZtq2AuITloHIGTR7HUo5E83QUV0Hv9ULUj6xvHNql10iH57Hj+er45acKakQq1LAX2kgq3kefeb80Np0aWmJZrOJ53lYlhUZFjA6dFUpteub753o9AZ3ASHvFSHvJX1TSrGyssLly5cpFouR6Ry3UroW7nuUiEcLZ4dJPeyFRqPB/Pw8nudx7ty5YRR+kIJgb62OXk8xA4p3exEoLnqLgeKiv1YPyKtiI0wDs5jD3Wrhd52BVC6WrpACdzOZe04j81Tf5PjnBv4XET8GKVBdB2GbyLyB0DSULhA+6FNFfMdDdR0cDYz4lIzpUoLQOl4yivZSzkv82FwdRLxQV80nIvcIketyqJ3WZkrBg0WBkiJQQGgS1Quc4kReDv2JYSAETJCvTJJvWoNIXGNuapixcyPLdnRmoiZBk2gzJbR7JjAMg4mJiQiBhk1fIVGPTsfO5XL0ej1WVlaG1qZxcr4TvZDhLiDk3bCb0iHsNFtYWKBSqfDWt741ER0eVimx0+TpUNN76dIlisViqoLhVjwMms0m8/PzOI7DuXPnEhHEbgNLd8LWj67hGRLN2b4xjckiTrxDSyS1xwBC13HCm1wEKQi9XAgmiYyQkjFdSo50mkzK57RqHq8WMw6q5hNRdJD+iOmFp4J0hXL7Q1MefbYcGLePQJUNUDrSMgIZmhRIy0TMDWoMKvhHUddQtht4E3sevuvi93pgCEAhhESZGrofeFEIKUFA3+thW/nA3EcMgg5L327tVqA8L/DnyJuBE57rgy3wV2Lpltly0lcjoRlM785Te7VG6/sk7X7gIyIKFriB+ZJarKMA69mzpEEIgWVZWJbF1NTUyCEoOp0Or7766nCqS2eQCsnn81iWxSuvvHKool632+W9730vvV4P13X5yEc+wh//8R/fcqc3uMsJOcwhjyI+nSOtwSGEruuHai6Jp0z2Q8QhDqOWiKPVag1z4ufOnYtMax7FuC3bTrPDxnfnUZ6PNlNBk5Le6hYq5bXfnK7grMWITROo0RtaBcNM8RVevYPMW+jlPHg+aS9CaVI0aZt4sSaJhL8y6emP1LetlC5Dra+g5+GHeXABvqZFcryiYKFiBThjqhjrHPRRRRu5ERxvSHfSAq818jkhAuIfyVHLSi6Ziy5YeLFcfOJNRZeYMTmgrCQ7GsXI8QQbFymt0YUkaYfXj6EhyznQNVSzC+3+8HzImRJ+o4es5oPPjAEhBLZtY5omDzzwwMhufdrtNqurq3z729/mhz/8IR/60IcoFot88IMf5A//8A/3vQ/Lsvj6179OsVjEcRze85738MEPfpDPf/7zR+b0JoQQKiX6ueMJebeUxSihjk7nmJ6eHnov7AZd14dP34MgjHLHIeIQR5GyCOV67Xabs2fPMjU1tev5GjdlsfG9gIwF0F8N8sVeTsd3+ug5HUY6w1qtVkJeRkqnmD5ZGOZK/XaPfrsXdHs5HtpEEWnpqJ6DV+8kVRgki1yBMiMZmbvxZVLgpVl8xqJoWbKHo4aGxzyVVgi0cePz51Jynrqjoo0khsTuR3+Drgm5XuxaSOvOi6s3UgqIomzDRvIcRSBTyHeikGxSUSOkbWhQtEHX6BlgOR5qvYWcLSUeTGF+XLtvioMgraAnpRwa2n/+85/nve99Ly+//PIwtTEOhBBDhZHjODiOgxCCr3/96/z1X/81EDi9/dEf/dHYhBwScRoZw11AyLshnIt35cqV4XSOd7zjHft+zThsykJKycrKynB69H6IOMRhUhZKKX7wgx/QbDY5e/ZsaifhTse7X0L2un1q319ILM9XSjjLNXyCwp00jKBYFiMGRSBVi5egpJHSNDJRpL9YixCwPldB+iowo+85uFvtYFJ1XIWR0mGnTxUTqoY0UtWrhUS6QivYuDFCTisEJkx7BAnCF0Ur6WtcySUmNhcrJbxYKqK3WY+aBFkptpaVXGI95auYeZNIutWleIDEISw9SNXMllE9J9ATGxpqpYE9sgcVU2VQMLe7886ML3eD/VlvKqXQNI1CoRCJpPcLz/N45plnmJ+f57d+67c4e/bsoZ3eQjIWQtwHvAvIEdwKLeDHwNU7npB3IhrXddnY2GB1dZX7779/x+kcu+GghBxGxBsbGyildm033gkHIeRut8ulS5dot9s8+OCDPP7442PJ4MbJIa+/ciEyQRqChhBvpPDWH7Tq2qemkJYOPoGvhVJYU2XcjRhZaILe2lbCpNtJ0Q8LT0U9MqRA5A0MswqeT3ezgeaq9NREiqoh9TSltTzHXdsEqQ0i8WXaZDxdEZC719xbV+xvxUi7aGPE8sKurWHGIuteuxuNuDSR0B7LlONKRMyaCIz1Z8ugFH6zF4yGikTMIpnesQ1ULJ8v8xZ+s4+cLIydrgixV9v0uHWQNGiaxquvvkqtVuNDH/oQb7755qG3OSDj/xE4A8wBUwSvOXngUWDljifkOFzX5cqVKywtLVGtVpmbm+PBBx880LbGHeMUKjYuXbpEuVxmdnaWkydPHnh69H4JudfrcenSJTY3N3nwwQdpNpt7pifSsN8ccmOjxtp3LyTuW3tugu6N9eg2LYP+Sm3o+CYtA3OyiDR01Eb03rdnq/RjNph+wUTE86KWhrsebw0WOIu1ISlogMhboBTGXBXlecHQz56T9NHQZCI1ga0nVB2h+c8oUok2pfVaJJzPUqRtukzIwwK/5RipFcyE7tuWBj4j29MlWiu6/a4JuU6UrOKXiDAkKIbkq7oOwjLw46ZAhZgk09RQcevQso2/GhuUOvC80B+Y5qDYrxfyUXTxVqtV3v/+9/PCCy8cldNbF/jPSqn/FP8fQojZu2JiiBBiOJ3j29/+Nrqu8+yzz3LmzJlDm9TvZ/3QkvLFF19kbW2Np556iscffxzLso5sFFMa+v0+P/rRj3jllVeoVqu8+93v5sSJE+i6fqCC4F4pi263y+uvv86P/+OLyHIOc7o8jEaEJunHinYA1lQpYr/p9xycRpfu9TWUIWAyjzFZDJocmkm5ml1MpnhEynQMt2AkIjStYOGs1HGWarirwfBPfaaMLNkYcxWMuQraRAFtppxYdzh9JLa9xLGk5IX9lHSFH09XFKykB8VEIREhixSLyXhOVuSMRM5XmyggYsc/mi7zdEHXFNRbTboFiVvQwZRQtFEbrWBiyFoT1ewlI2ZdJn2bK/mECiOethHlwcw/cfD8MexNyM1m81BeyKurq9RqQWDQ6XT453/+Zx577DHe//7385WvfAXgwE5vSqkvjZKxCKAJIaRS6u6IkOfn51laWuLMmTOR6RxHMYZpt/XjEXE8NXGYydW7FfUcx+Hy5cusra1x33338dBDD0WkdQctCO5EyP1+n4sXL1Kr1bhv9hQby20cL4jHjHIeVxfYeRtvKdoIEgw2TfosGKUc/VYX+h7+epMeYMxVkEJD5szA68Lzg8nOMakbArSun3BU01UyGuq2OglLTuH6uButSBpDny6BqaHlLYShIzQBmoY2UUD1HPyOM3hVjz0wpEhG23krpRMvJYou2ngxD+bUkVFxm8uinTQFKicHlyqlotNChED4Pk7BRPQcdC/IzQdpB5/Q4a1n9CLaaSXB24gODZDVZIEv0UVYMBO2ncIyoOCjnaggcweXi+3Hx6Jc3r/ZfRyLi4t87GMfw/M8fN/nox/9KD/3cz/H+fPnD+30JoSYAv4HghP+PaXU1xkMVBRCPHhXEPLs7CwPPPBAQu97FINK04ht1KS9UqnsmCM+zOTqtH07jsOVK1dYXl7mvvvu49lnn03tVjqo41t8PcdxWFhYYGVlZWi8f+0fX4pEvE69DZrA6TrYc1WU6w1zu9ZsddgIMtyHZeCsRokbQHjQWw+iEiEFxlQpKKCtNSJ5W2O6jBtXNKSMOXJ1gdGOnntXE7Ae66bLGUE7tgKvHxBpMJgzZuIzXUK4Pvp0iVanQy6fQzMN6Ac6Y+V54PjBhOiuEyHXVNe2uCeFLpOa6onCdiSqSYShoZVsfF0GKRApAxc3Q0ObGbyJOB7K94P0ga9Q7T4K0KYD8o1kz+P8rwnsWIrcK1roMT/qbrsTUcwI20imKwpW4HynS2Q5h9Jk4NHcdtDmDk6WsHfb9GGNhZ588km+973vJZYfkdPbHwLhGKH3CyFOKqX+avD3f7wrCLlSqaQS0FE0dowiTsS7aZghINWD6phHCXk0L37vvffuOaPvoIQc5pBd1+Xq1assLi5G3jpa11fZ+lHS81gORg+1F4P8sVHKo5VyePECGINxTjGS1suFSIFO+Yr+egOt3cPv9NFKOfSCHTRCpOQFtbyVIGTP0jBiUVtupoIba+fuSJ9cnJgKFsQIWWoSdxDlWoDfaiCnS4m5ckKXCKVQUgT/bWioroNWzm0nbC0D4bhok3kYTNEIp0Z7rofnuhiaBpa+PS3b84MGk0YX1eoNuVTkTIgPPJ0t47djbybxYqEuE2kUOVlM6IpNXcdn5FxYGkYnel6bwiG0oVIC+pbAliJoEql3UBttxEQ+UFfoEu0AZkKjcF13V1uBw04LuVUQQpSA/0YpdX7w97cD/6sQwgH+CWjeFYS8U/L+qKw5xyXiEJqm0e0m86L7QRhdX758mZs3b3LPPffw7LPP7hoZjK570KaS9fV1rl+/ntif8n2W/ssPEp/XC3bitdpptNHLOXq1JtZUGWno9NfqSE0mmkMAtLyZaB82Zys4KwNt88CwXisG2l69WkDaRmBc1Owm1BogMHopColW8gFRkCb+iEm7ApzNqLG9SpnhJ0wt0ekny9vpBOEr6HuBif1a1KBHO5GiwpASfytwmdMBJfzA2H5E6SErKd4YJXs4YmqI2INIpHlQVAuJjjrhq2jQbGopueJCtBtQE5TLJfyCh993Ec0evq9guRFOpwoOyffQAO3eyVST/XGwH3P640jIQBmCyqsQwlRKfUcI8VHg/wJOA95dQci3CkopHMfhhRdeoFqt7puIQxw0Qvc8j+XlZZaWljh37ty+iTjEuDlk3/e5efPm0Iz+2WefTVzwyy/9kPqNFfInpqDn0h8oEKxKgc5SLJq0DfqrdVCB1wWA0DXskxP47T7Oen274GTr9FeSJK1S/JH1Yg6n2YuoH4wTFeh7AUG7Hl69g17OJ9Mak8WEC1o4yDSyLMUBTpVNiPl1OLaGUY+Z2OetpI9GWl44bVpI3KFtqpjI00rbxIsRsh8n45QCn5zIJ1qqiT2whaWnFurixkH4CjldDLTXPRelFGqlgYDBQ0ygWdGuQqUJ5CDtcaG/Rv+7W0NXt/26u41iPznkY0rILvAfhRAnlFJLQghNKXVVCPFLwDeB9Z8KQt5rcGfa55eXl7l06RKe5/GOd7xj3w0doxhXS+z7PteuXeP69etMTU0xOTnJ/fffP/Z+95uyGJ0cPT09zSOPPEKj0Uhc7N31Oovfeh3l+bRurAJgT1ewygU6MZkbQG66mlgudEn3xjrK8ZCWgTFZpN/pBdFSLxYdT5cS5kLC1BIkC0EuNkGCukYvr1EoFlB9D6/RQepaohAojOTln6ZRNg0TLzbDT4v9rApw1usRDbWw9AS5axP5ZHRcTnGAi0sbUkzmRSlt8khyW/SjB6tMmXwA7ES+kwXQtUCF4isY/C7DNu+ZMn5j+4GsBBidmNplsoi/2kAULd7yXz+F4zi0Wi2azWbE3c227SFJF4vFVNMg2B8hj/pfHCOsAp9nECUrpTwhhK6UuiGEeCfw390VhLwb2YakuJ8n8ChBTUxM8La3vY3vf//7YzeUxPe9F0bbuk+cOMG73vUuXNfl9ddfP9B+9yLk0YGl4eRoy7JYW1tLiOqV77PwTy8ORzKF6G00EK4f+FZUbGTXQ3X6mJUCncWNxD6tyTK9wXK/59Bd3ETYBuBgzpbBV/Q3msFrc4qu35wsRyaFwMBwKEbcsmjjLNUwAWcwdVnYBn7fRZ8tI6RAOR5ez0m2Xpt6Ut+cMxMezI4pMGMaXz1l+Ghb+uTjXyZlkKeKaZaVxu4FvnBZzkw8jOIRs8ibiYhZFS3Exgi5a4GpkZwOXOKUG7T7q3hKZrYUTWkIkXhIeKVkETBs6dbPzgCBx0y1Wo1EsUoput3ukKjX1tZot4PjzufzQ5IuFAo4jrPrPdloNA7ce3AroZTygSUhxMNCiF9USv17pVT4Cp0Dvn1XEPJuCC049+rsiRNxmJoYx1M5jr1SFmGq4MqVK8zOzvLOd75zONo8nBpyEOyWQ15fX2d+fp5CoZBQh6QR+dK33xgW60ZRPD1N+/pa8JduHw/IzVYxygXcuLtY0aa3vJnYhixYqI02vQHRCkPDPD0Jro8w9aExjpDJHO5O0Is2TtzsfqKAs7QVkboZcxW8XjMwuDH1QBZmaHj1TpBrHihJ9Eo+UQj0DAmxrrg0JUVRtyL5aV+Av9aIdiIWzCSpFS30rVh7dkreNRExl3OJiFkWB2oHIRB5k54KCobabDlQh3T6CE2iVmKR72wpOaw1PsFkMo8fM+5PdOuVBpNVhEB/cCbxHYbHLgS5XI5cLsf09HbTSGga1Gw2qdVqXL9+na2tLb7//e8PSTqe9jiuKYsRQ6FJ4MNCiBbwzwRden8AvHlXEPJeBkN7WXCmEfF+1t8Lu8nmFhcXWVhYYHp6OtVf4zBeFmk55FqtxoULFzBNc8eRUHFC3rx4g8v/+l2Kc5NDJzcAe6pM+2aSpKWu07i0iDC0YPClL9A6LtIyEi5kejmPHzO5UU4QZTvrDRBgTBSRtonUNXrX1iO/s17NJ6JjYSY7+NKmakAwPkg5Ht7mtiZZK9qRUVDC1kEEvhnBSj5e30Xr9oKockA+wtITc/NkOZ8gTGOmnJhS7UgSpksJkyBdJlMfk8Xt4axSICwDWbJRpj7sClS+D55C5MxgQki7j7Qkcq0VIVs5W8YfbeEWKQNQJwuJiJm49jtvoo1qq3UNWbIhFxybOID2eNQ0KMRLL73EE088MSTqMO3hui6f+tSncByHubk5Tp48ySOPPDIMcvaDa9eu8au/+qssLy8jhODXf/3X+cQnPsHGxga//Mu/zMLCAvfffz9f/vKXxzbAH7RO60qpF4UQvwv8H8CfAteAP1RK/ae7gpB3Q5pJ/Sgh7kTEIY6SkEcfAFNTU7s6zo1rhTmKUUJuNBpcuHABgEcffXTXDqZRQu7WGlz4+/8PlKK5FJCvPVHCrhQRXTelo0zS3QzIUDkemuOhADVTouc5CF2gudvrpA0hDXLHA0JV4Gw0EVIMxz7p1QJCiCAnbBoJm01jqoQTa04xZspJH+XpFCOhqVIkNaG6TqDqiG1Pny1jbnWDA9QlwtSDBpKug9AG7bqDaJucEZjED/yRhaGhTRWHfskKyGkayjARBJ0CXacHnqJX0PAGv4UwNUyhoWkSKWRQELV0KJhBftj1gwaWtWagiw5/z8nCUB8c5qQ9XWCMPhuFSKhk0sZAJfvkUwqRRYue8JGmjuGLQJ632gDXR3v8aMegmaaJaZqRSNj3fT73uc/xe7/3e2xtbfHpT38a13X5m7/5m31vV9d1Pve5z/G2t72NRqPBM888wwc+8AH+8i//kp/92Z/l+eef5zOf+Qyf+cxn+OxnPzvWMQ8iZFcI8Sjw74BXCJpCvg3Mw13i9rbfCHmUiCcnJ4e5091wFIQ82tE3mrPdDYeR7GmaRqvV4tVXX8VxHB566KF9vcKF5kJut88P/+ZfcDuxhoDNBlaxQHurQfHUJF6jgzuQq7m2jhHLqwop0fo+bqODEiCnyghdw3c9VMyfQQFuLzl5xJzb9rfoD0YwhRpnY7YCQuE1u/gdB3czGQknok1ApCSp0xzbUjFaIHN9lNdHSRGV1Gky+P1GJGuiaEFMNaHNlIaFtPCI9KqNHneTMzQYycs6EjREpDVaThZQ8XFOmozmfCUJDXHaenF3J2HryXxyOYe/1UEWreBtoe9CvYvV86Ad7EPOFPFXm4iihTxxuGaQyPHscG9IKTl37hyu6/LJT34ykvrYL06ePMnJkycBKJVKPPbYY9y4cYO///u/5xvf+AYQWG++733vG5uQYWgz/V7gPQRR8QUhxCeBvxFC/PZdQci7IfREvnnz5lhEPLr+YZpLHMfhxRdfHEu/fBh0Oh2uX79Oq9XiiSeeGKvaLITAc1x+8LdfR5k60tDxR+wTK/fO0RyoLLauLgfrlG2EoWE1HOJhVO7kJN2wwKeCsU0IgV0tIqeKuMoPbmpPoU8XE8SghAiKh7Hj1POBR8Wo94V5eiLIPcs8fs/BrbUGUW/sVb+cSykEWgkFhyxayTREJZfu4ha3/JwqJZQOWiGlVTruJicEMv5QK9kQSx9Y0+XEuKh2s8Vor6jSRaLIKCeLiYdCooJqG8n1Sjn8fhNRsoMBtUqhui6i76EGaSc5VUD1osepusG1o52bPdKegL3QaDQO1akXYmFhge9973u8613vYnl5eUjUJ06cYHl5eeztDYp6AP+glPrfAIQQllLqs0KI7wG9u5qQlVK0Wi0WFhY4efLkWEQc4iDtz6Oz8lzX5Z3vfOeBHN/GQej4VqvVmJ6eplwujy/98RVbL/wIZ1DE0m2TyqkZ3FoLq1SgtZRUT2geiF4fV5eUZidx6m28Zhdzokg3pZCXPzVF7+bIdgRos1UMy8Sx+xHFgZkyPsk1JWKlnvD09TbbEYWBEoEdqDZbDlInA1N7aRtJU59iDjdGllrBTi6zTFxiVphpXsgx1USa+b0omIlXfm06SZgyZybyuWlKilysFdvJG4OH5MiyTtR/WuTNIaEO91ex8TcGI5dMPXB8CxU2jS6qAWKqkCgeJh4ulVxQBNTkUF1xFNirbRqC1MVBlVEhms0mH/7wh/nzP//zhC/GYZ3klFJLg+1IpVRPCHEP8B2gdlcQcvzk+L4/nJdnWRanTp3ikUceOdC2Q5P7/UApxcbGxrDB4sknn+TVV1+9pWQ8ajQU+k2EPtDjwOu7/PAr/zokYwC322f90g3yUxUMTaCXcjgjxCKkxM7bdNfr4HjUB1Fz/uQkWs7GbcZm2xVt+isxHwsFmq7RGSg2jIkiWs7E77oJ6RcEBvjxaLaXk9gxQnItiRbL/8qihd91BvK3YNK013cS2wuKg7GCoaUnP1cw8WJFRK2ax4/P9EsZgJpmLqTiCgVNJPO0E8ntD5UUI7BVNF2hckbCjrMtPExbQ1gGmqEFE0C6bkCuW10Ug8i31ok+AOOBfdFKKDCEHuxfe2A6IPYjwl4aZKXUof2QHcfhwx/+ML/yK7/CL/7iLwIwNzfH4uIiJ0+eZHFxkdnZ2UPtI4b/CfhZ4D/cFYQMg9dtzxvOy5uamuKZZ56h2WyOTU6j0HWdVmtvyVVIxLZtJ1QM4zamjGKndUf9LeJGQ+O2Tveabb75v//feH0Xv2giRyruVikPrkftymLw98kShVKJ7nKN8ulpWjfWEtvTDYPmwjLS1LFnK+D69Dfq6LaF04q5ktk6vRGSdjabOJtgn5jE62kYk0WUr3A3WxilXJIUpSDva8T93/yUm1Ir2Lgr9UiEbJys4rb6yJKNtI1AvmbqqHYfv+eg2r2gflct4C7H0hDFHF7cCjOFLBLTQzSRSKWIsp3UGU+V8ONNHnH5myBJ0BP5bWN4Sw8It2ihSh6NepNSLo/qOeRdheh7MAg4WhYUetGmFBW/jgpmwkhI2EZg0xmuY+tB5G3r6A/PJc7HYbBfc/rD3G8f//jHeeyxx/jd3/3d4fKf//mf54tf/CLPP//8ga03U/YVntzfB/4M+O/vGkK+cePGsONsVL3Q7XZvqQXn5uYm8/PzGIbB+fPnI/Ic2FYujNP6HCKtqcXzvGE3305GQ+O0Tm9eX+aFL/497Y3tG780O0mxVMJrdQBBfyTS7W006G00mDp7b+BPkLfx2tuvr4VT07QG0a7fd4fyuOJ9s+CDXs7hjhCiMHWITc0wJorDAt7QoEgIhG1gnKig+i7OZguhwJirJhtGJovIWGTp6wJnNZbq0EQQCfsKv9ENUgNSIE0jUgyUeRNcD32mhJCSeqNBoVhAoIKI2PEC/whNpkfMcdP5FKKVtonXiDVUxNuiw/yuoSHCPwUTHH+g7iBQv+gaFAYa454HngrSHK5PEVC9NnK6lDASKubyqN422TqmwIhFx64uo0oAU0OF59rUgmaYiQKi0UWWc8jK0b4d7hUht9vtVEnnfvHNb36TL33pSzzxxBM89dRTAHzqU5/i+eef56Mf/Sh/8Rd/wX333ceXv/zlA+8jjoHyoquU+vd3DSFLKVP1vGmTp8fBToS8tbXF/Pw8Uspd5WQhqR6UkMNId7Sb7+TJk6l+EyH20zqtlOL1f/02N35wASemRGisbOD0+ygBxXIJ2Y8W96pnTrB1+WbwFyHIzVbAB0vqw7FNozCrBbo3N4av5EalgFHM4eMPGxKGECLZNgxYs5WIS5zQJfpUCaRAn0K/wTcAACAASURBVCzg1NqBoc8OsKaTTm9u0cSIeVSoqo3ajBv45CJFujyglSVeLGLW54p4/WagitACi0yRN5GaDL5XYOyGkAJtpjR0f1NSBKmb6RKdbgdTNwJv5o6DquQCG03XD5QNK3WE74PjE3TgClS9s+3+ZhvQc0Ftl1jlZCHZFh2fRl20ohPAAataDGRr4bnRBXLkodHXwbXBxEB3FbLjYgNivQWOh/bO8WfZ7YW9CLlWqx2qoPee97xnx5THv/zLvxx4uxAYChH8LB6B2sIHxCBS/poQ4r+9awj51KlTO1pwHtQCM1x/lJDr9Trz8/MopTh37tyeP/5hOv3CguLa2hqXL19OdPPthL0Iuba0xjf/z6+y+OOFYD+6xtyD9+LVO/TqLYxqAb/n4PddtuoddNtk8vQcfqNLfqJMczRNoRSd5RrC1DGmJjCny3iN9rBbTxoaAhH1UN5qBZ69PRdlauQny6hOH3erjX1ygv5iNOIVhpZwg1Ouj5CSflgglAJtIo9etFE9D9cAffCzy4KV0CILXWJ2/UQrMLE8qxLg1FoRJZhPinG8oQU65oFNJhB04S1tRXyHtaliCpFXhg0jQcnZQ2gxhzYpBrnc7YeVqOaS3XPlXMJISMVsUEUlh4rN6ZM5A39UbmdqUambqQWt1aEVaLOH5XhYXYEYub+ahkfFgX5eY1O1KTYl+Xx+V7vYcXAHGwsB/AfgCsEYpwbBcNOGEKIBPMTdYr+5G45qakij0WB+fh7P8zh37ty+f/SDdtwppej3+3z3u98dNpHsVyGyUw6502zx0v/zL1x7Yz5SaPRcj5s/XkBIQfHkFEXdoDtSRXe7fVYuXmPmwXvo9nvY02W6IybzUtfQ8zbtlU1CCsnPVtFNA1M36cbVGVKgWyZOq49w/aERkTVTQaHQp4pBU8iAyKzpcmLWnl7J44y+9vsKt9ZGuD5esxuoCSwDrWSj5W1Ut4/X6g2N4fXpciJiNlJUHdpkAWJKhI4FxZiSQpssJqRuMk3qFg++UvwgyCelZ9pUoOmNrGpo0c1pyQnSMsVjQ+ix9Swdf0RyKHIGYiKPcBX4fnDOui5stCP5cDkd808WkPMD4vXOTdHr9VhfXx96UowaBxWLxQMFKbfanP5WQQhRBD5CkC/WgWngAYIXrgnAVkqpu4aQdxOLH6bq2uv1qNfrvPnmm5w7d27sdslxCVkpNfSbcByHRx55ZOyKbjyHvLW2wRvfeoVX/9//Qq+1ffNPnJghl8+zdWMFs5ijPFlh6+YqHWD6zCks3RhK3WbP3svmwuJw3fxkheJEhf5mg1ypSG89SkbtlRrlM3PUl1bJzUwglaK/FmiKCyen6MZM6pEC4Su6N4L9CUPDmCghLQO3liyqSl1LFO6suUpEJaJ6Dipn0L+23eYtDA2tGjj3GXMVfMfFb/cDb+W4lEsIRNdLRNFW7Of0BfTX69FpHGnt1JVcUuo20hwy3EXOgHYsUo8PRM2bybFQU6VkdOxFz1FfB2ujBQKEbUJOR+TMoLjneAH59pPkK2aSOefEbL+pIuZaE1G2qTx2hurIPen7/tA4aH19nStXruA4DoZhJDwpdoumXdfdVbV0XAmZoN3mfwb+F6VU5CIQQujAd+Eu6dS7FWi1Wly8eJFut4thGLzjHe840HbGmasXFggty+LJJ5/kypUrB9JTSilx+n1+/J1/47X//CKXXvshyvcpVsvMnTvD6uXr+J7P5tIqNSm455EHMTSdjYWbw22sXQ3+e+LUHJNz09QuL0X20d7YotdoUT05Qx8Pv2BEmhpK987QHqQ2WjeDf+s5i+KpKdy2k1CP5E9ODd3gIGi/7q/X0Ys5vFYXvZRDK9rg+khDT0S30jaS7m2A1KK2m8rxkJqWjI5PVlHtPqJYAhlMXhZGYBqPFMNWcX2mhIiRnjlbSUTHbc0j34tPSY23wCVJzdXA2IwXAYuRCBYGUrdI0U+gYkVBqjbC9xGTxYG/haLfbpKT9iDqdcBxodlHjfgXi5kiajUa+cYLjGKqAPFGnnYfR4J9/mQiQJJSUiqVErWWfr9Ps9mk2Wxy7dq1oaIpn89HImrLsoZKqjs0ZdEkIGQDhvlkjyADZgK/DncRIR9VJ1C73ebixYu0223Onj3L1NQUL7zwwoG3t5/Gknq9zoULFxIFwnGj69rqOld+eIFLr73Jmy+/iue4TJ6cZfa+0yxfvkazVqdZq2OX8uSmqpQKRbr1Jjd+dAmA4kQFI29Bo4fnuFTmplHO/8/em8VKup/lfr//Nw81T2ueulf33tuzwTY2iewbJN8cCVAQMhcQySJBoOhwrggXUQS+CLmxiJAV6UQHcZEIrCARHCHhgOD4QILZ2wZ7s9lTD6vXPFWtVfPwjf9c/GvVqlrVvXfv3u1gtvNKfdE1fFWr1qrne7/nfd7nSdh79W3sjEd5ZYGo2SUeBFi+i5fN0Dm6lhR65TxeIYdlGAwfYz6kuzb9wwYyTjB8ByPrEg8CPN+bAeOrchYKEye4eLymrbsWQoKed9EdC5kkxO0hZt6fJIxclVHNEd9ctMg8hk82VfqHnB50aUIdfwxEwjbGdqEw9ASZXHY8lpGAVGkecaKATQi8MGWan4gdHaM5mBlXJlkbcQNEY0fH7N/wL75aujA0RTdYyp9YK/sqV08IpbwIE6RjqE53FKPpBrLdB8aRT7ZOJhJIef2aWml2cAcgb9AsopSZNxYad8/Ct8C1wNKVdtkQ6OtPv5BkWRalUolS6TrWKU1ThsMhvV6PdrvN0dERQRDMzINM08T3/Tn64ocVkMeDu0MhREkI8ZNSyr+7uk8IMQJehw8QIL9TXV3CvxP3NBwO2dnZodvtcvv2bSqVynMB+XcC1V6vx4MHD4jjmDt37sxdar2TnjiJIk73jjh6uMvxwz0G3R4Pvqf8k92MT6ZapHt+yeXJOQCVlUVSJINWh0KtQv+yzWV/RGV5gWGvTxLF9JptaKrli5UXt+gdNwjGkregN+D43iM0XWP5xVtYUqN3OKvvHly0MT2H9uEZWtbB1HT0QYyGQHMtZBBOtr7i/oi4PwLHINJN5UuRpGPuWKqA1Bu8MYCZcYkaXeIgJkaBhFnNEY9CjFoekKT9gHgQzA6pxmU49twGnlHOPoZPzpPM0B8xesEnOe/gAslQgbrxmO7YWMgr8yLbmISROlkHhpEazklJmiYgIHKnBrC6hogSpG9OTISEbyObA/W3mEhIYvS8O0NzSJQP9PTPK7LOPIjmXLTG1G2amE+GvskLw7Uiw9CUe5ujwl2JUkWtDCJldDQIqS8Ksk/rC/KE0jQN3/fxfZ+FhWsdcxRF/PM/qxixo6Mj+v0+aZriui6ZTIa33nqL4+NjPvOZzzzT6375y1/mz/7sz6jVapPXeR4ubzBjvbkC/LdCiP9JSvkfhRCLwH8JbAP/lXiP/Or7W4H5AVaapk9UU3z3u9/lox/96GOHYqPRiJ2dHdrtNrdv36Zarc4B8d/93d/xuc997pkAen9/H4D19fXJbdNd+J07d2a6g+l69OgRtm1Tq1Y53T3kZGefZv2CR6+9yfnBCYsbKwx6A1rnihJY3FwjiSIax2pjLlvMU6iVOds9wivlsC0LTQgMw+R872jyOm7WZ2FtmcvDM+ycjxZL+s02ummwtLVO0h8xGDu5LWyv0zo4I41jstUi2WKBUb2FDBPcxeLc4obh2vg1xSHLm965uoZmm2ijqctlU8ddVHafSbNPOroGT2epNKc5FoaO7lgzydQAYcYkY7toloFMUpJBgOE7cy5vWsZW/OyUZE5YBhrMXsZfdafTvKqpK2XGlOGQcMxJKOnktpwDNzrhqxTomar4iBufUWBrONPUh6Ur7dx01l4lg2zcoDRuAqulq4vjqfel3aQmADwFrFi6Wp92TUSUqs/oil7J2DAN/mVlUCQzNm+vpnzs4x/nB1WvvvoqL7zwwsQTRko56aa/9rWv8dd//df0ej2Wl5f51Kc+xe/+7u8+9bH/5m/+hkwmwy/90i9NAPk3fuM3KJVKE5e3ZrP5LKZCMFYhjvni/wL4EvDPwF2U6uI3pJRnPxId8pUF5zQgX3k/NJtNbt26xUsvvfREwH0vqSM3a3r1ehr8t7e3n9iFD3sD9t68z2sv/wP1vSP8bIbT3UNG/QFC01i7ewvbcTje2UdoGusvbtM4PuV09wCEYGV7k3a7hZfN0bpsYmdcstkMxw/2Jq9RXVvCtGzOdw8J+gOiKMLJZ4jTBDnWHCdRzOG9HRCC1btb+JkMjXt7E+lVt96kW2+iGQZONY/1OGc1XSPujgg7faycj1vIIgcBaT/AyPlzsi2ZpITt/iT12Sz4GL6DkPKxAalWNTczyAMwihm01oCk35t4HQtdQ5rKKlOzDCQSGSYI05hTQhhFf94c6HGGQaV5CZuen49Q0ixzNrmZ+e09Mb7kn3le0cO5wSf3jQR/6ibJtYnP5FhZe67LFQVvFnw1oYzjTQ3hKd8KaeqIUYxMUZREe6Qc7aY+nyvwvb7hmgdPXqyij+apqudZN2VvQgg8z8PzPL7yla/QbDb51V/9VV566SUePnz4no79+c9/nt3d3ZnbnpPL26TGSyA7QBuluPgPUsr/+ur+HwlAnpa+hWHIo0ePuLi4YHNzkxdffPFdO9+rwdyzALKu64RhyNtvv83FxcUTwf/i5JwH33+dN779Dxw/2CNbKmC6No2DExqAm/HYeOkO+289YP+tB9iuw8ZLdzi4v8Pxzj61tWWWb20wGgw5PzgmGgW4rouMEnqtDt2LFsWFCpl8juMHu9QPTsiWCqx/+C4yTTl86+GE39QNneVbG4T9If3LDkvbGzQOTjkZBWTLBYq1CoOzS5IgQrNNvHyOoN7hEshUCmSLeaJmD9Oy0IRG2FFf4LDTJ+z0EZqgsL6opGqOjn7VIYuxk9mUeiBq9Qm7QwzbRCQpZslHt0zSYYjmmHNgLHQNEadzqyVm7Xox5AqkjbFhvLANNM9WHa+hQ5woRcQwhDBBePZcZy08a86fQmTsudu0gjevhniMskIr+o8JIr3xQ9gGfpTO3BH6Bs4NaiYWcvaL7ZmIOCUtugRRiO+6YOrqiiWW0BkhhQDHnFFzzA3uxLzaQ1RUly3KPnHVwzi54VXynOvdqMdOp0M+nyeXy/HJT37yfb/e83B5AxgHmiZCiH8D/Fvg/wT+B+C/E0L8r8C/k1J+cEJO380TeTgcUq/XOT8/Z3Nzkzt37jy1WP1qMPdeneLiOObs7IxGo8ELL7ww95rhcMQbf/+PfO+v/h9OHx1guTa19VUQgs6FkoVVVpeIg5BW/YL9tx5QXKjiZfwJF1ldWcJybA7eVt2A6djka2UaByfU947RTYP1F2/TOD6jedYgiVO2PvoiQtN49NpbdC/VF6hQK5MvFTh5uE8SJxw92GV5e5Oi7xIMR4SjAAF0L1p0L1oIQ6e6sYxvOvSnIp56jRa9RovCSg3N0tENE4bXkUhC08gtV+gfXPPPVs7DzmewHIfgsDG7qScETjEz8X6IGl0iILE1zJGOlrPRLRORSOL2ALs6v5FnlHziGwM/YWqk4+UVGcQkQQxCeVbMWGwaGkbORUYWwlAbd91Ol1wxQzoIkFEKUYwME3THmuu2hbjB82kCeSPiSrjWfMJ02Z+jeLRp7lgTCEvH9VywJEIXpEgSmSLDmNBVJyYjFoS6xG0N0VGiVxkM1RubpmQq88oKbsraph9jGwjfVtLAgov+iTXiOHxuCyDvVO/0XW+328/E8T7t674Pj4yrDzsC/nsp5d+P//9LQojfQ2mU//0HBpCfVFEU0el0ODk5YXt7+7HeD+9W73W5JEkS9vf3OTo6olKpUKvVWFm5TkwYdHp898+/xT9969vopkmmpIZ54TDg8O2HlBeraIbB+f4Rl0dnOBmPu5/8CEEQcHla5+jhLku31um3uzTPFLD5pTyGadA+u6Cxf4yd8aitLFE/OiGOYiorS+gbOud7Rzx89U0A8tUS+XKRowd7tM4vaJ1f4OWzLG2t0W+0OL6/O3nP+WqZbDFP6+gcwzIpLVS42D3mEsjXymimgTFU7mm122u0D84YjIdVhm2RW6ygI3BNey4ENewMcApZOo+O0R0Lu5RFA5LWALdWmN/c0zUcxyXtDpHBkHicHBJmTGSnh1ZwiIIQVzPUpXeUzk0/jFJ2bgnEfNyAruDP0RKaLUiObyyqlHzSzhDNtcDQVXqI70AQoVeyVwwiwjaU1CzrXu82mwYiTkAqL99MJqP085WM4ouv/CmGIZptKt+MVCJy1zSERB3OKnjI3tXfqgDPwrmhae4ZCZnRlIGQLuDmxt80v2xoymBfE6prHkbqX16Deg+xVkSrZEgbjWeyCHgv9W6A+LxVFs/L5U0I8XlgR0r5f928T0r5b8ePER8YQL75i5p2Q/N9n62tLVZXV5/p2E8LyGmacnh4yMHBAcvLy3zuc59jMBjw6NEjAKIg5B+/+Z84ub/L/ts7RGEIjBi0u6xub9FttWk3Lmme1jFti82PvkC/0+Xi6IwHr76Bl81QWqrRa7Y5GfPH5bUluhdN+uNOd/3F2wih0em06be7OJ6H0DT23lQxToZlsvbSbS6Pz2nXL2nXL8mVi1RWF0mimOMHe+y8+iaWY7N0Z4NuvUW/1aZdv6DXbrO4uYZpGIw6191b+3xsIFTKU91aIhiOSNNk0unGQUj/ooXr+wybXTLVIloiiS67CAS59YWJTC4ZhQzGuuXsWo1oGGAs5JFj03mBwK7kiOrzzmteNOa/BzFK5JkQZAzMNEbL2+iahhZLpVm+yfN69ly6NLo2v5AhBPpjtu3kVZRSrDwmpC7QwmSWArAN5Z42nSSSd6Hem5wvfEBkxRx9oVX8WTmaPuaAp99GyZv3N/asmfcQm4JMdBVcoWpogzdQCzCxKcDW0JMUo+SrDb1BCKYB074jngmNPuga+sfU9+rd1prfbz2NAOFZbQqeVM/R5U0H/hshxCFK4nYIDFELIwlqa2/1A6OyAMUPx3HM/v4+x8fHrK6usrq6yvn5OcPh8JmjwR88eEA2m52R4EyXlHKSSFKr1dja2pr8YQ4GA95++22Kust/+t/+lN4YOP1CDjvrcbyzPzmObhisvLBFGIacPjogCkKldLi9zsG9HZKxZKy4UCFB0jqpY9gWy5trWK5Dq3FJ/VBt0+UXK4hY0qo3xs+p4uUyHD3YBcDLZVm5tU6apBzv7BEMhji+h1fME7S7jMYbfUIICss18oU8zf0TwtE1X1lbX8FxXdpH53jVArIXEA4VSHiFLIVambQf4Hoe6SAkGswCiOnZmMUsOcslvuzO/HVlVmszZkIAmm3ir5QhSkkHwcS4XRg6pu/O+V1YC/k5TjfRBUIXSF3l4WmmjoFAtyySRkdJy67e30Lh8ZK2p7hNX8iRnt1wfqtmSW+qPArezGAzEcq+dNr8RxTcuQ5W1LJzxkzCtyfD0OvnzX7mQcbAHiRKP2wZKs06lepKYhBBkhJmTawpc/vQBDNhJjKKogfNAdpHltE/pDjWoyOl3Jm+GnyeFccxr776Kj/+4z/+2PullHz+85/ne9/73jNRC7/wC7/At771LRqNBgsLC/z2b/82P/MzP8PP//zPs7+/P3F5e5Iq6p1KCPFLqBXpH0PRFl3UX3wO+BiwA/zOBwqQ79+/z8HBASsrK6ytrU0un+r1Os1mk7t37z7TcXd3dzFNc+4PTUrJ2dkZOzs7lMtltra25s7O/W6Pv/gPX0f2QwaDwaSbvKra1ioXZw0s18bxXE53D8kU8zhZn+OHu5PH5cpF3JzP0cM9TNemvLSAbdv0mu2JzM3N+lRWlzi4v0MaJwghWL2zReeiSeeyRXV1iUwxjzbmj9MkUbri25v02x1a4/emmwallUWiIMBxXC6PTkmTlNJSjWw+R2PviGSsxKiuL6NpOlEc4WnmjI2npmss3FonHgb42QxJe0AyXrTQTZPsQonhOFVEt038agFd03Ecm/B4XoPsr1ZnQFpzLYych+XZJJ0RSefaKjJyNJxEm5F5IcAs5+Y64TBr4vTUyS41BdgGuu+o3DqEWvgIYtAF9MNZiVzGUl1kMn2brcBt6nHa44yAHrOO3HcEmWkMFag06em1bm/8mtPHn6YYTHWyETlXDe00EKkkSCL0XjQTNituenVkxqA+9U1P8w7aVDfetyROJAh9nf7Ha2RyWXzf5/DwENM0J0Ow511BEPDWW2/x8SfI6q4A+fvf//4P5PXfTwkhXpZSflYI8e9Q4aYpUABOpJT/OHncBwmQDw4OKJVKc5dNrVaL4+NjPvShDz3TcQ8PD0nTdKIlno5oyuVy3Lp167FZea3TBn/1v3yd9pkCOk3XqWwuc3BPASYoEC0u1Uil5ODeQ9IpAKmsL9G6aDJodxFCkKmV8HyfcDiicaRWmYUQrNzZ4vL0nF5LgWG2mEf3bOIgpFgtEwchhmXSrDfpXCj+tlAtky3lObr/CJlKEILlW+v0BwM0IQi6A0b9IUtbawgEp3uHE7CzPYe1u7cRiZxs+V1VeWWRbC6jVpwT6DWmulwhKCxVyeZzWKmYMSi6quLmMsOTC9xyHsu1SYcRaWeIv1KZMxgCcJdLE6WFMHSMvIfuWPR7fdxEU+vB47KWinMDPz3vKXP1KXCTugBdoEdTtwm1fCFGMZGWYvse6EJxuuPfmUyVGQ+WqS7zk1R5SSQxmmerjl5K9S0ydWWGP90JZ9VyhybE2L5TIPIeDEMVFaVrao3bMhCxcqoTV+RxlCg9dKCA+nELHrGrY0yFnIqiBzdkdeQcmKZCyj60BuokYxnqxcIY2Rkx+LEF2lZCr9ej3+8TBAGe51GpVN6XgdCT6iqO7cMf/vBj7x8Oh/z0T//0+9qs/UGVEOJ/Rqnafh61Qv0GY7c3IAa+K6WMPzAcMihZyg/KgvPKsWo6GeRjH/sYnuc99jmnb+/yxl/9Pf3mdceYJgnnDw+oLNSQmsBwLC4Ozzi5r/TB1cVFpA7nYx+Jxv4JmmVQ3Vpl1OnRO7+kxyUIwcaLt7k4rdNrdTi8t4NhGmx86A7BcITl2JwfnWDZFmmScvLoACklmq6x/sJtOpdNWvULWvULSgtVirWyUqEcnRINA0pLNUqLNY4f7nEyplQKtTL5SgkhYdjts/P9NwCorS/juK5aNEklw24P13dpHBxTWV3EXyzRP2uOgUhiOTYXe0ekSUJ2oYwE3ERJzfJLVYZj/4thvcUVVBS2lomSGL2WI+0MJ5l17kp5ZlFExglxe4AmBXYvJkXZSmq+g+FaECkJm+yPlDrA0hWfe8NH2SplSG4a6RRdRGusjU4gbQ2Icib2Dc5Wr+VIT69/5wLQFnLI8y4aQt0iQCuO15GvhmBSommaeoxEvS/DQLSGs2qIsoc86Uw6I8l4CWSamrD0+Uilagbjhg6Zm1roio/sBgqor5JJQuWtrFKvA6hkoDNC2yqT315herf0/v37E6vNaQMhy7JmDISe1Y7zabyQb+bf/RDV/wj8LHABvAB8EsgDLlAFPsGPgv0mvH8LTl3X6ff7fPe730XX9ccmg0zX7j+8znf+6M9Jk5RCocggDhhcXndmwWBIoVpB6Bra1JrpFZ2x8eI258enmL7D8LJD9/AcJ+Ox8eI2+/d2kGnK0f1dDMtk88N36LW7eJkMF8enJFFMbWOVaDhi1O3TaTQpLlTwsoo/Pnj7IcWFKnc+8RGSKGLvrQc0z+qYrs3i+iqne4dcnpxzyTnZUoHSQpVwNMJyHE4fKXBe3Fwjn5Rpn19MTh5Ozmf19iZBu8fpeAHldOcAANt3Wdxax7UsWo+OJ8O+zsk4WaSUJ1soECSxMoO/4oKFIL+2MMnbuyoz7+NXCqRhgrCn0j10DbuUncnDS4cRmm0Rn7av6QtdU7acvqvsJT2bdBQi+8FjlRZ6KYPWvhFblLGxbngnhyZY9dmUbJF15rbhRMmb6161anbOU0LLOLNLGIaGvJlKXfTm0rpF9sbzHHM2dkkIRDWLiBLwx1JOKaEXKte3cPz5V/zZbjnnqBBW10T7+PyAPE1TMpnMjAXAlY1sr9ej2+3SaDQYDAaT9ehpoH43n+9/zV7IUsoD4PeEEN+UUt570uM+UID8JCL/alPvWarX67Gzs8NwOOSTn/zku1r7PXrlNb7z9W9OJsLDlvqCrr9wm9O9I9x8hiQIuRgDme+7VFeXOLz3aHKMbruD73n4+Sz9c0UxjHoDTh/sUV2sodsWp7sHLKyvEPSGBJ0BmVyWYDAiiWMO3nqA5TmUthY43T2gedbAtCy2PvIiQsDOa29N5HL5xQqGpnNxfMbB2w/RTYPl7U0EYJgGjeMzoiBgaWsdL5ehda700ADVtWX8XAaB4PjBLjuvqq65tFgjW8jROb8kGUvuTu7vksYJbi5DaamKFqYML9p41SJiENE+uBbc2zmfXK2E57qMbtIUQoH/cP9ax2xkHIyspyiOzmgiAQMVJiqCaDZANEmVZvjkpnQtg+wH6KXMRHOcSqlOIO5YrSAlqQBTCEVRTN6XwHYcmOJ6UwFxEGBPN+CGBsMbnalvzflOiEpmHmgLN4Bcn+9yKXkq+cOzEJauhnamuhIIBkPMRKDZBtR7swRk0Zs9VsFVKoqr0oTq2k0N7dMb6uriRj1uaUMIgW3b2LY9k4KeJMnEjrNer/Po0aOJ1j+bzU5A2nXdyff6X6sX8nS9ExjDBwyQn1RPE2l0swaDAQ8ePGA0GrG6usrFxcW7/rKPXr3Ho7/9Ptlaic7Z7PDu4tExK9sbRHHEyZS+N+gPCfpDltZXaPd6mI5J71Q9d9jskC3k8Qu5CXXQqV+yuL3B5kt36Fy2uTxV5kFHbz8iW8yTqxQ5uLdDOBgxaHXZfPEOCMHeG/eoH4xPAvkshufQOW3QPlXd59LWGrbr0u10uDg+JRwGLN1aJ1vMcbZ3NAHhZGEpzwAAIABJREFUhY0VTMtG1zVG/SF7r9/Hdh28ch5LM2ie1rk8Pad1cUlhoYKJzWCoOqKUhGGnx1GnR6ZUIFfJMQhHFHJZkovOBCBM2yJq96kf1BG6hl8tYrk2BLHi0G+AdDIKsTMewb76WYShE5hjbbamITsj0lE0+WJbS8V5bXHGVskaYXJtiSmE4pjHl/8CwDQIrBTLNNCqllqwEMrXQgbJ2DNCKtD3bfTWAClTNfSTkpEJ3hSvLcevI6ZPGJY28ZNAU1yy9CzFy5eUwxsaCmijRA35xscXo1iNi660wpUMnKrO20Zx4ZjGLBjf3Mgzx9y7Nc7sM3R1ImmPEJtltMXH0wLvJapM13VyudwMxSClZDQaTew4z87OGA6H6LqO7/sTwH5Sp9xqtX5oO+SnrR8JQH4vEpjRaMTDhw/pdrtsb29TLpcZjUbvujJ5fm+P7/3vf4FMUoSusfzCFqcP9iZDuuqtVS53j5CpZOX2Jq3GBf329SXqaDgi67h4xRxhs0c49r+IekNavSFr21skGgw715SAEIKNl7a5OD2n1+zQa7bptzvc/uhL9Ieqo77a+PNyGbxijsb+MYN2F9pdirUyhcUa4XDE6e4hcRjhF3OUlhY43zvi+KF6ndJilWwhT5IobfHZ7iGW61BZWSBbKtC9bBGM5W6llQUFToOI1tH55OfTdB2/nMcyTXw/w/CsycV4pbpLA9OxKS0vkMvnSC/6hGNTIZmk9E4vsHMZPNehc9bAKeXQTRNGETKI8fKZGVMjGSdoQgFIdGWfaeiIrIOZdRFRil70SPoKgIVtKDVFOOvKZ1SzcykdWsElc9FHDgcTTNMqGeTxjU3AcgbOujPRT1oth9HogaYpYASijI7VUy5woEA/dgzMyXJHquKTuuFsB1v04Ia3hyjfoBhcE24kUid5G6MVzD6mH0DOUYGzAvVe2iMF9q0RFMfdcs5BfOzJkrZnzY6cvH8hcF0X13WpVquT2+M4pt/vs7+/T6/X49VXXyVJkonLWyaTIU3T/x+Qf9jq/dhlhmHIzs4Ol5eX3L59mw996EOT472byXz3/JJ/+MM/n1wWyySltXtCaaFKlCYEScTl3vGkK2kdnqGbBsXVBZpH5xTXFuidXjLoDRk0O+pSbWOFo/uP1CDMczBNk+buEdWtVYL+gGFvgJSS0wd76KbB1odfIE5jWmcX7L2uroqKtQpu1ufo4S6DTo9Bp4eXy7CwsUIYhMo97vwSN+Ozsr3J+f4x/WaHfrODm/GprS8ThTGaNlZZCMHCxip+MUf7/IL9TheEYGFjhVE4QiBonzVIkxRN16iuLmOYJpfHZwggVyhweXxGt9EiVy2hC0HSHkAqMT2boD/k4PAMoWnkFyt4GR8tiMnmciStPmFLgePVqrZdzOK4NpFM0WtZiBLSzggj46CNohlTdRkn6LZJcjTbXWu+jZ51EYDMOiqhOYjRfGtOR6yVfbhBLWhZZw70hG/N64az9jUtIaWy1ix6OJPnjg2b7JRsb5aGGOkSd5rmsNXW3szxH2MYj2nA+ESJqdE3JI5UAz41OEzU67aHECRAoLrlaX2zYyipn66hfW5zbHT/+Hq/gPykMgyDfD6P7/vkcjkqlcqMy1u32+WrX/0qf/u3f4tpmpyenvLxj3+cX/zFX3xmlcc3v/lNfv3Xf50kSfjlX/5lfvM3f/M5/1SPrw+U7C1JkicO77797W/zEz/xE3PT3SiK2N3d5fz8nK2tLZaW5tMO0jTl5Zdf5nOf+9zcceNRwGt/8h85fuPhRMo2XeXNZbq9Lv1Geya5+aoWXtykc9ni8mi+A88vVugFI9JhwLBz/SUxHZvCao2jB7vEYcTS7Q36zQ7haERpdZGjh3tEwfUX1i/lyeTzCEPQGiszHN+lurbM6d7hZAnEsEyKSzVAYtk25wfHxGHE8q0NguFwsnQCsLCxiu06JEnC5ckZQX+I7bnU1lcYdnoTH2Yn41FdXUImKTKVNI/PZ/hcy3dZ2lhBxqkKP53qAnXbpLhQIe4OyVVL2KYF/RA5DMmuVEnbwxmLTABvoYiBxiAK1OA1iEm6Q5zF4rx5j6Fj5ty5wFKjllODOFNHOKbiSy0DISGNY4b9IZ5pI9MEDTGb+mFoCNtUIDb5hWmIKyncVTkmQsoZ606yNmk3mOmqRSUzw+VKYGhLvPA6sVp6BrphKGmcpikTIENHBBGEKQQR6BpxnGBMf1xVH+pTIO5faaqn6JOKr2R0d6po6++8EPGd73yHT33qU88tLOJm3bt3j1qt9sQu+Ktf/SorKyvcvXuX1157jV/7tV97JjVHkiTcvXuXv/zLv2R1dZVPf/rT/NEf/dEzy2bH9VQfyo9Mh3wz/TlJEvb29jg5OWFtbe0dPS7eKZfv9f/jW1y8tUuhUiRKU7rn1z4N+ZUq7SMFQPlCnpCUfv1al1vdXqN+bx8ErN69Rf3whGBwDQ5u1qdXb1HdWKLBKYMxKEejgPqDA5bWVnFyGXb+SUU0AZze38P1HDLlAu3zCxzfo1SrcPbogEwxT7FaoT/exDt46yGmbbHxoTsMewNc3+Ps4IhgMGRpa53SQpXT3QMO7yutcXl5AT+XRQhBr93hfO8I07GprS9zeVZn1OlxMOaa1164jWXbNE/rHL19PbC0XIfKxhJINWy9PDjh+Op+oXL+Mrkcru0QXXYZnauO9nJfnQyEoVNeWaQfjHByDmIQTlaDs6s1knqXWEosIOyp57qrFaW2qGYnpjoyiDAcax6Mq7lrVUQ0TgDxbcRA0SMCZdBDHKL7aognDEPxuaaGcBXXKzx7QksIw4AkUUsXVxNHXZvljYUgBUZJqE4kEqSuoaUSqhnFS6cSHINML0CKZCKHS8IYo399Iot8Y+b/APg2RnsKjYveLBibmqIv3LHSIUxgPPwT25V3BePrH+MHA8bwdCqLz3zmM3zhC1/gC1/4wjO/ziuvvML29vZks/dLX/oS3/jGN94vID9VfaAA+Z3qCpANw+Dg4GCy0ffZz372mS+zDl7+Z+pvKjAZXnYQmmBhe53zR4c4+Qyjy+6kGwzGfOnS9gYXR2fkl6tc7IxN4iU0905wXZvyyiJnj/aprK9Qf3gIwMXuMYZlsv7SNscP91TXeneTxt4JraNzFpYW0RxzwvmGgxF6nLB6exPDNDm895A4immdqaFXebGGl8ty+PAR1dUlouGI9uk57tY6ju8S9IeTIWJ5sYZfzJPEMTJJObr/CM0wyFQK5Gsl2ueXHI0VItlqESfjEQcRh1MLI9XVJZyMx+VJHdM0EJrG2a6iQCori5imSfesQTRSbmGj3oDznQNs36WwUMU2TZLukGwhjx6lROdtIuAKTjTfJl8pMQoDzKyF7IWIVCIMDbd6PcC7SnfWPAvTMpGDEC3vjrlTgWboyH6gFjCu3Ok8S7U2U517KsDMjdUMoB6bpGjZzIzZD4w73NMbXG/Fh2k6RAjIqm24DAKGfXAMtJSZxRFK3uR5V7Anyh7WlCtc6hjoo2SmHet7Ar89IhYSzbPQHFMtpVT862DTjDOrqij7SolRzSAeI3H7l6inUVk8Dw756OiItbW1yf9XV1d5+eWX3/dxn6Y+UID8bh3y0dER5+fnLCws8NnPfvZ9GaH0603u/8Xfz9wmU0ln/5Ty8gIYOs3W8dzz2gdnLG6vEQbh3H3RMKB/2mD9hW16ndkvcRJGXDw6plytkqkW2Xvt7UnX3h1vwxVrFTTLwHJsWqd1LnYV4OdyOXK1MocPFcXRbbTIFvIsr69h2jbHh7vK02Lsc1FZXsD2XPqdLtlSnvrBCWmakquVsDyPUbc3GdhVVhbxCznSJOHk4R69RhNN1yms1EjjhE5d+WsUFit4hSxRGNLtdjFti6A/5GxXnXQqq0sUVjPoQjAac8VBf8jZzj5u1qe0UOWy1SRbyGOWfegGyDAmt1zFCFPSRo8RKnpBCkh9i1wuRxiH6Fkb2QsQUrm36XGqtvMA2R6qxbnFAvKse7W6AY6JlhtLroRQ7mxpShJEBDLGuGmhWcvO642rWQVqM4/LzN1G6Qb/qwvlzTydMpK1Zwd2oBQUVzI4S3XommcjUqkGcxKkIfA6AVKPMROIByHhKMROrr8raclFm15wyTvQHEDBRfvJLUWF/BDUv2Yd8tPWBwqQH1dSSk5PT6nX65RKJT796U8/M9F/lZQspWT//371yYy6BsOTBpmlMoPzFunUQLC4scTluDNe2Fih1+1NnNoMyyRbKXC5o0Bq9c4mp4cnxEP1xfRLeTQhaNzbY2FliVQXM1FMWgo5P0OapHi5LOHYzGfY7TPs9slmMpQ3lmmdXXIylR7ieK4a6h0eM+j0EJrKc9MQaEIjSWKiUUhjT51gFjdXMW0LoWs0T+pcHJ9hWCaLW+tEYcD53jHNozMs12H9xW2SJKXf6dA4OLnu3IQgVy3h57OYusHZw31aJ9eqjCsjfNfxGJxe0BrrlPsX6rOyPIfFjVUCGSNtAw0TMaYu8qsLyGYfrTXiys5dAknewSFRXaRjqFDQYYhVK8yDqW0iejeUDZpAyztk2+PbdF1JwwoujBIoZyYyOEwdEadQy4KUSImiNKJEdbpXG3mOoXjbkkeSpGr9uJhFRCnY5rXqAVSW3ZUdp22o7TnTGK9Np+DZszRE3kVcjBRXDaRIjJyLMbXVN/Q1tO6A0JJITSB0DTsM0U2B9pl1dfXwFPUeZ1HPVE+TOP08vJBXVlY4ODiY/P/w8PAHZph0sz6wgCylpF6v8/DhQ/L5PCsrKxQKhWcG4yvKwzRNjl5+g/prO/j5HOiC7pRhkF7KMDxVHWvc6JIrFxjFEaPLDpmFEt3j662z7ukFQtdYurtJY/+EbDk/2V4DaO2f4uo6me0NRqMRQavLYAyyVx4R1dUl4iRGN00GzTYX+9eDt9raMmma0jg6JVsu4GV9Tt/cwXIdNj98ZxwLNSQYDDm+t8vS7Q3SpZTWRZPWiVq86F600E2Dtbu3GXS76KYaIJ083Mf2HCoriximSat+oVQhwPL2Jo7n0r1ssf/mg8n7cTIelZVFolGAYZqMegPOHihqxMn5OFkf4oRRp0+2WKB5VOe0P0DTNUpLC3gZjzSIKBSKhM0unf3ZQWimVqJSqRAlMaGe4ms6IpYIQ8evlcZLFfH4H0Q64BqI7gByFrppqHBRTSg97/TAzdAQWQcxzTlLqQx8zm52xtfa36vSKv7c46Y1wqD8GYUrENPPNTQFvtNDwryjlBDTiykVf5Zy8C0YBArANYG0DdoyoGDq6rGJBFPDafSUsg6BtHTVY4QJh5s2F/ffQEqJ53kzyxqP+w6lafoDN6e/aoieVJ1O57l0yJ/+9Ke5f/8+jx49YmVlha9//ev84R/+4fs+7tPUBwqQr35ZFxcXPHjwAM/z+MQnPoHrupO9+metK+lbMgjY+9Y/ABC0e8owZ2OJ1sEpmm0iesFM4zwah4Pm1xeJ+sOZbhmURK69f8bi5upEe3zzfqIEW2p4SzXOHu7P3D9stKjdXlfvbRQQ9K8Bo3/eBAEv/PjH6Fw2J910OBxx/mAfyzJZ+vBdhoMhSRJz9uj62KWlKinQPm0o5zgJhmagoWEYhloM6Q0mXPHC+gp+Icew2+f04XX37RdzFColOo0mSRyjCY1+q0OapJSXF3B8l8bhKaNOnzSKKa8sksbKsEZ4JlpskgYRjcMTqutLiDhl7/4DCgtVvMUCIk6JOwMWN9ZIm30GU2vWQyC3voBvOcRpjMg7iCBGjGK0oo8Xy7H5kIQwAiIGLmSDscubAGkbiKyNpinuMil5jIYjPNtGOBayPVDKhvFQVVRnVREw5oxv3KYUDrMAnRYdvOYULaErg/kZqiJrXzvOmZoaJmYddfKo+OPnaaprFgLE+MTiWxQaEYzG7yPnQGd4DeqGprr9RKL/Z1tsLebYQgHtYDCg2+3O+FPc3KjTdf1f3Jw+DMPHmny91zIMg6997Wt88YtfJEkSvvzlLz/R0Oh51wdK9ial5OWXX0YIwfb29ozfxNHREVEUsbm5+UzH/qd/+idu3brFwTe/Q/2NR3P3m3kfO+fT2p3njQGcSh7CGD3j0jw4nbmvvLVCa1d1ttnlCr12l0FzfGlezCL7wURSZ+Q8NFOneXyO0DSqmytcjF9TaILC2iLN8wa9ZhvDsSguVGkeqtcrrCwQRAH18dq26dpUV5ep7x1RXlui025P5GrqgIK1l7bV53f/0WT5A8ZqidVFBr0ebibDxckZw24f3TDI1krEUUSv3iRNlW2nl1UudY7v0zyv05syXcpXypSXFefcPDln2J0Fr5UXthAS4lFI2O4jp+SFpZUFHENl9wlDxxA6Gc1Ei6G2sawij6b/ajVBZqWCFiaTYZ5IJDKKMF1n1ooSCBwNM5IY6TUYJAJE3kWbNvQRQCWD6IcKSDUx6UzFFTiOSxoqXmnyf6kSQeIgJAzDa8MqgVpXvqIpTF1t3wXxtcn9TbC3DfXaU6b08oa8TfrmONVEm0SBkapsPe0zG2ir73zZL6UkCAK63e5kq67f7xNFEQsLCxOQ9n3/uYL0K6+8wmc+85knvqf344X8/0E91Zv6QAEyqAicx11SnZ+f0263uXPnzjMd9/XXXycvLXb+5G8fe39+Y5HeUYPMSoXLveMZra23XGZ0ei13y6xU6V40CboDihtLc5feQtPIrdUY9gf0G60ZAJq83toCUtc4v783d5/QNQobi7TqFwwv55Oa80tVDM/m4vB0Dvz8SoEYiZQputBpjgFaNw0q68v0Ol0ujk6xxzrm+t4RmWIeO+NxundIdGVgP/Zi1g2DYa9Hff94hmcsLdXIlosITePwzYfXVw5CqDDWQg4hJUkYz2i0haZRqJXJlgoYmkZQ78y4oYHyaHYxsFwb23Mx0NDDFMu28Uxr1lsYlVWnp2MQcwyVF2doYx4YRKKsLeUoIjYEiUxx4uvL8xRIciZW58YV2OM64xu3SZhfXdYFZKxZc/mMWh2foVGmh3oAto40NAXWlgpslbauPJFTybDTw/U9pYe+Oo4ulMKiM0T7ic2nlrfdrF6vx+7uLisrKxOQ7vXUe5umPLLZ7LuaCD2ukiThe9/7Hp/61Kcee/8PsxfyuH70dMgAjuM80YLz/Tq+Xbx8D6uYIWzOXmqavsPgvIlMU7oH52QrJZI0oV9v4pbzBDcCNntHdQzLpPDCFs1HR9wsmaYML9sgBEbOI2rOdnmarqMh6B7VWb67SfO0zrBz/YUurizQO75ATxJWX7jN5em5WpdGmca7GZ/Go0Py5QKZUp76/onqwlDyPKeUQwYxbiFDv9UhHI5Iopizh/sYpsHdH/socRhxvKMWUJqn9cmxV7Y3sWybTnM2k8/P5yguVhkNBggh0DSdgzcUv5wtF8lXigTDEReHp7gZj1G3T+u0junYlNeXMCyTYauH0CCby9HYOZh8Jm4hS75cxLYtfNNF1rtEQhB1+vQBYWiUV5aQnR5JxsXIKy8OLQXbcxH13nUaxtj4XWQcxNkNMC24GMNQKTFyDlLXSHUFFkkU03chTVKEFGDr2L0RumsgUiBNID/e9BOoz1sIZeJzoagvCSRCDd4YhIqu0ECaOiBUh3w1NLwC3rKnZHdXoaVXCdRBrDL5Tqb5aGbAWOpjOiVOEZ/dRFt7NjAG9RmYpkmxWJwZrN2kPHZ3d9/VROhJx3+ngV4QBM/Ve/lfqj5wgPykej+Ob2ma0n1wTHJ0AQJKW0s0988mXbBXztM9vHYfG401yaVbq4SD4WRpY6akypTLlQvEUtKdMiMSukYqJbIfYABetQyGTvv4XNESqzXah6pzbe+fYRg6y3e3aByeklso0z48nTiRtfZOMHSN1Rdu02t30A2DxljF0W+oxYlqrYpTzDEcDAi6A4Zj4/ig08fWDbJrSwT9AZbnEnYHHL+hEq5t0yS7WiIKAjqNJgsbqwT9IZd7J1iew8aL2wwHQ+oHSr2RqxTRpWDY7VNYrLKwsUr94FhlArbaLN/eoLxUQ0PgjimOQbvL2aMDctUS+WKRbqNJEIY45TxpGBF2Bpi6gWvYtPfO6AK6ZZIpF3A8h4yXwUUf0wuCuNVX2XxZG99wCJtDFd7pWWi2ie7a6ChnM5mxIIgRsYSyj7joK9EDQDRC5hy0foQ+3bXaFtLSlEJjXLGQBIbEm/bbuFoiuaJIpCS1dWVENEWbyKIL7WBqe06Oh4FTVz6eOV7hm6IprgzqXRNh60hTJ+p2cX1b5f5Fseqg6z30z269K03xbvWktWlN0yagO3lvNyiPaROh6U7a87zJMZ9G8vbD7vT2NPUjA8jP0iFLKTk5OeHRzg76gzG3KqF3UCdbLhDFMULTZsB48txUoguBnoJbKTBszHooZJcqdA+u+drK1grN4zpJEOJWZ7vqKw+H2tYqWDoXDw5njpXGCe39UxZvrSBTSc80iad0zmmSEnb7uLqF7toMsx5h9/pLP+oOyFVLBIOI6vICl7pOb8xhyyRFDGNKpTJREKDnfILBEJmkJFHM8LxJrlqi8uIdhK4xbKv3Gg5GExOkzZfuoJsGvWaHy9M6SDmhSlzPZWl7gzRJOd87nBlKAizeWsfL+ETDgNZJnWgUMBo/18tlWL67hYwSUg0yi2WGzS5JEKJrGq4wGRzWGQBC17ELPl4+i+/4aGFMEiTKGD5VGnChgdsOmCZARNZGOBpxu09oJIoCMQ2luohSpGcibUMBpqXDKEabAmOZsdDTFG8q+Tk0QaYJ9hQlEfsmWhBjTsUrycqYyhjfJMUYjPuBGu7pmnof2tgK1LOQSaI66dZAdeHDEGm70ByQjQVy1Ffvc6zc0P/z22iL7x/I3ouPhRACx3FwHGfGRCiKognVcXh4OKE8fN+fDNWjKHos5fFB0CDDBxCQ38kT+WkBeVoyVywWueVW2BvMRhWNLjsIXSO3uaSUFDe4eCvj0j+9nPhX2NU8cXtAEkZkl6szYAzQO6xjmQbZD61weW9WSXFVhmnSPTindmuN1ml9ojMGpW9u752CBN9xsVcXqe8dIeOEbK1M1B8SjgbQ6mIJQWV7k0G/T++iRWGxSnM8VGztnaIJwdqdLUajEULX6DXadKfkeDk/i18r0u92MRybzvE5w6khXW11CcO1GfWHWJbF+cNrTWehVCJXLTEcDBQN0e5y+LqiLoSmUV1bxvZdwmGA7VjUd49oXikBBGpDsFLCsR26J43rbcdxZcsFFtc20IWGNHSsco6oO4A4IZfPIzoBQeP6ZxG6RmatqlJAkAwz6tJfRCmxJcj0QgQCAzAQoKcgk5mkaIRSV1xpmdPxarQsuEoRIYSSoQnANTGCcX4f6uor0oFRREICukQTGtLRsTpDNFNTlIeG0iFP+3EUXbgcXJtaaWMKZMocSJY81Tl7Fv1gSCafVQM8KdE/vYlWeXLQwnup52Es9CTKo9/vc3Z2RhiGvPbaa8RxjOM4k85b07Qf9rSQp64P3FAvjuPHOrNJKfn2t7/NT/7kT77j85vNJvfv38d1Xba3t3Fsm1f//Tcm8rXpyixXGJxc4lYLjIZDgvY155hbrdI7upF04TuYeZ9Rs0vUG948HG45R9gZYOczpBozIJhdqjI4b06AX7dMvIUiFwcnZGpFBmfNWcN0AFMnt1ajeXA2WS6ZLivjka0UCIKAi4OTmd/ulYIj6PbRPJuz3cNJsCkonjodhRi2iWabnB0cTxKndcNQAaaNNkbOA03QOb8kDlWXWF1fgkQy6vXJ1crEcUL98ESFqmZ9aitLNMY+GV4xx3A0UpLBKGb51gbt47q6AhCQKRVwcxl0XSPjZ+kfzJoXAZTXl8hYLrqho1vK1EcGCYam4bkeaXOWK9YLnjKl74WkOmAbJEISItGkxBA6hq7SqoXQ0W0deTm4ntpoAlHyZ5ZNJEDFRzb6k8dJ5iVxqQDyDlrr+mQbGBIBWPHUsaoZBfaGhtA1pK6WOmQyjqSKU3AMlUx9NR8wwdGUCsP4/F210PKc6gedOH1+fk6/32dra2vON/kP/uAP+NM//VOSJOFnf/Zn+cQnPsHP/dzPkc1m3/Pr/PEf/zG/9Vu/xZtvvskrr7wyM0T8nd/5HX7/938fXdf5vd/7Pb74xS++l0P/aA71ntQhv5sUptvtcu/ePTRNm4lounxrbwZopw5IMjYaH9ZbaIau9Mh7J/i14hwYA0T9EW65gJPLEMfxJBsOFAAKIZBJymisjChvLNNpNEnThLDTn+nCkzCie3BOaWkB3TYZiBZz50tTp3/YwNFNvNsLXB6eTqgMN59BE4LOeAOuslhD9xzqe0fohk62UqK5N14yuYSM45HZKtNrdfDyGS53T2bej2/YLN1dQeqCTv2SwThNOhoPQH3LwV1ZIBVqIebqyqExXu/2bIelj7yg6JfGJWmSTsz7hSZY2d4iHQUYmk5pZYHeZYtBq8ug3aVcqzKqt2idddSJqlbEdCwMTZDzc4yOLwi5/qw1y1AnlEaPIB2iFxyEoatMO8MgbvfRwxQdgZ6AkBqWY+E0B6jvldr/C7IGZi9UMZVAqqmVbUPoaKMI8u61kbxkEj4KaisOKdVJtOwhJQyCIYbQsIVQ3LEEbANrECLjhESkanHGEXhTnXLkqCGvNlKfqWTcsV/5ZegCmbFJwhGimEH/sTXEVXTTc6okSX6gQ7VpDvmmb/JXvvIVPv7xj3Pv3j2++MUv8v3vf/8d7XLfqT7ykY/wJ3/yJ/zKr/zKzO1vvPEGX//613n99dc5Pj7mp37qp7h3795z115/4AD5vdZVMkgQBNy5c2eOhzr5u9fxKwVGwxHxFO+aXa7QP74exKVxwuCwTm6xAsbjwd8u5eiNqQrT0PE3q7T2T5CpJL++MCd/6x030HUNp1YgOJuXr+mORTIMGJ5dki3kMLIel3sKKIVjIBLVKaWk9A7OcS0Ld3WBUb9PMggJe1M8crMLzS7VxRp2PsOM2IKGAAAgAElEQVTFDa20jBNG9RbFSok0SVi4vc7F/vGk60WCbdu09k/wXYfqizV6nR4XJ2cgYWFrld7ZJWkQknc8vLUiUZrQPG9geS5alNCYomryhTx+uYBpq7y71v7V+7mWD2595AUsbfwlzWfpX7RIwohRq0Npa53ovMOg2UCYOmbOR3dMXMfFFPrEhzkdBKSDAHshj+yMkGHC1VcsNXWMWh4RxspfopaBFCUJtEyczhBpaMhIGffLnIPRDhAympwaA0/D7IA2pTsWBZUkPb2WPbDBjUBLU642Can4k+GdQKlFyDl4LeW/gamReCZpkhAlKbGjaBdd1zEHIwzPRItSpQhpDhC2QP/MJsJ+/l/7H5QX8vTx322ot7S09L6d3l566aXH3v6Nb3yDL33pS9i2zdbWFtvb27zyyiuPteR9P/WBA+R364Sv1i+DIGBnZ4dWq8X29jaVSmXuue2dY4Z1NYwTpo5eypBcqo4vHs6bAwEYlsHwvKV0wPtnM11kGAbXl6txQu+wjl8uovv2YweDAO5CkeC0iePZWMUKzb3TyTH9Up7+OO4p7AwIOwOcrIe0dfSYuc4+CWPCVh/bNDAXcvQumpMhHCj5mIwSurunWAJyG/8ve28eJNl5lvn+vrPnnll7Ve/d6k2SZakXSRDGZmwjLsbYeO5cmGHiwp35A/7AYjFBEAQBhANMzEwMtgxhM2EWQwQEBDgChu0ax5jBHsvXapnRYkm9791VXXvlnmf7vvvHl+dUbtXqllot09Ib0aFWdmXmyaWe857nfd7nmSMMQqoLy2TK2nMiiZcCyGey5HaP4XcCZBSy0e2ow1aH6hV9yb19906y5SK11fW0O5fdFBAnm2Hfvn1EnQAj47C+vJJ6VQgFRqyIFror4rPT2PkMYRAS+QHFfIHW9TV6iRjXdph9YDeOMPUwspTTiyRhjCGhYHrES40E7jRQj+URpkGn1cFyDFwvA36IcPSKuFqo9u+VTBag1iEON99bkXEQGRvTjxB5PWxTptBhqkFIYMbEsSKOY5QhsKMOlmdiZD2CwCdSknymqyhQoITQnXVX1qak0nrhINayPNuAUELRw1xrYSqwQQ/qHKub66dPAI0M5FYb1HOCixOSPfUNChRwHOeOLlC80YCc8MZbVbVaTe0y34i6fv06jz/+ePr/27dvT2maO1n3HCDfrEzTxPd9rl69ytLSEnv37uXQoUNbfjFvnDiV/l2FMaIaU9w+BYagcXUEgApB1NJbdc1rKxQmK7TbbaJ6G1lwsRrDsjt/vU7Otiltn6a5uk7Y3OQOvbEiftcPOGr5RC2f4kQZHAvDtqhfGREr1QkpVcpEfkBcEUQ9mmnTtbFtC7/a1GAtBBO7t9FptomCABnGyG50Ego6Xdph28E9IIzUkzipOIhQQYzV0V4ahft2U11epd3VPJfnpoibPrUL+n5T09M4pRzNegPDNonrHWrXNoebNjA5OUllxwwqilifX06JBn+jgV9rUpgaI68cRKz0zylFp9bAdhzGiiXC+WofSHueR2XnDKYUKBTGhEnc1Bl77kQJ1lsYEvTScQyGxJoqwkYbIRRiLNdVMhjdUNEYkXVQUYwKIoxu5p5aD5O3DQouIgLV8DHRHhV4FiLjdS07YyAmEBJlCQqhgFZLEyEZWyeR93hXiLEcaqO9uZ2HtvVUK42uy5upc/gsa5MWkQosg0IjQNxXJN7mYt+4QbVa5dq1awRBgOM4fTKzV9MC36zuBiDfKae397///dy4cWPo9k984hN8+MMffs3HeCfqLQPIUkrCMOTZZ59l165dNzWkB2ivVKlfHv7QWgtr5OcmyM2Op1FCSeW3TdDs4Y47q1qPXNo1Q21xbfChACjsmKLZ9V8wHIvyrjmqV2+glL70DAcGdf5Gg8xECTNWI+V0pZ0z6clCa5jHkLagubiOkXH6u2alaM6vYmYdTMfB8AQdP+znhqcq+MtVYj8k67pkdszQqjVoLK9T2TlDe3EdFUuito9fa2IB0zu24ZRyVK8tEfYoQfxak6DRZnz3HMF6HWt6glqtpukSqbAzHmPTk7Qua5DOYFLZNouZdZBKYYWKzqq+hFcNn7DhgxDkJ0pkYgMFONMlRAxhvUW2UsRTJupGvU/K5paykM0QNTvY5az2Ro5lN1pJbHoahzE0u13xeqtvI1A4JkY5h1praqtMz9bDvLyrO1mhtc2gjeaFVKBAjmdptVoYlkU2m8WTCoUijiWhipFhhIxjlAemIRCOjRPHmAUPoUAhtMxto60Hj4FE5D1UrQNR91RkoPnrpQYcnkYdnsasVslms+zZsyf93vdqgZeWlkZqgXO53C2ZBr3ZgHw7xkL/43/8j9t+/rvlAHfPAfLgGV4pxfz8PJcuXUIIwYMPPnhLH9zS/x6d1p2ZLNPqAnFp5zT1hRVk16tglHICqTCFiWVZeKWCNvzplmFb+GubwxkZRDSvLZOrlHDG8lTPD18SCdOAWNFKwkunx5B+SGejTm5mbKhzTzXM+3YQBSG1pt+nxlCGoQ3But67uVIBs5ChubiGV8oR1zvEXZ44OT6AbffvQwYRgWURxT30jRB4+SyNy0vYQlDcNYcyoLqwijAEhUopfYyw2cEG3FyB4s5pDASNpf6TXGe9zmRlO/6CPvGUZiYxPJsoCFFRTM7xNB+MhFC/BmkKnGIOK1CEnsIcy2BgQCfCzDmY1UDP2hBQ9VFGgDlZ1GvICnAd3dF6ttb6SgXlLESxXmPvhriqZN25uxgi8i5qoYfrFyAmCnCjtmlYD2THchjrLWjU09utyQLmck++nilQBQ+xvvmd6pgKYQicngstMZlHrbf0mnfOAdPUHLGUiMd2Y+weZ2VlhQsXLrBr16405kxvSxpUKhXGxvSGnmEYRFGUgvTVq1dpNpsIIcjlcn2bdYPgeDcA+dXM6d/IxZAPfehD/MiP/Agf+9jHmJ+f5+zZs1v6aryeuucAOalES3zu3DnGx8c5fvw458+fvyXf1jgIWX152EAItKdDUs3rK7iFLMKxkIKUXugtwzLw12oYnYigU6e8c4b64iqxH5KfHR9JfcRhRHt+leK2KTr1plZYdMubKuPf2HyeaFUHjVZ2z/V1o71V2DFF47KmN3KlAkbeo3ZdL2jkJ8r4K5tLKHHTJ276WDkPKfRGGQN0eWnnDM1L+vEcy6K0e0KL+pfWdQ5gwocrRbtLe5THyri5DJ0g2IyvTx5v+xT+/BpIpbv66XHMrKddxUwbf37zJBZ0T2ClXTPI1QaGI7DnNKBEzQ6mbeFhdXPuFDS03WbkGBiGgVP1iVxDb+XZFoZhYJqmBtfkqxHGiIKrDeJ7/ZCFTphmuaGR1dUmPRQ8vfkmBExpLawyQAgDFcWEJRe/08FxXdx8RvPAlRxpnpNn6Y3A8ZxeoRaie+Lt+ikDmAaZSCK7a9oqiukYkuxqA1MJiCOklAhLQrWF9R17iabznHrlFaSUPPLII322AnEco5RCKTV0W6FQSMHNMAziOKbZbFKv11lcXOT8+fPEcdznUREEwesKfHi1uhXK4k54If/lX/4lTz75JMvLy3z/938/Dz/8MP/wD//AAw88wA/90A9x//33Y1kWn/nMZ96QE9A9p0MGWFhY4OzZs+RyOa0l7g4Dzp49S6lUYmpq6lUfY/Gbp7n2T8/1aVqVa2KGamgJBECOZWG9NaSBLeycpjkAulbGxR4v0JpfHRmMWtg+SbPrmywMQVxwoOEjTQPDj0Z+CvltE7SXNsjMjNFa2Ui5aLecJ275QwZFhmuT3zFF7fINYr+f27ZyHrZlpR2/Wc4SyIhorYFRymLWhweahmmSn5sAAZ1Gk/bqZqeYm6oQNzqoJI5ICLzxIjgmpmXTGliSAb3daIYxsR/hVPIYjkXYCQjrbUpTY4TLw6qT/PZJ1HoLI+9huLYGnE6INMFtxRiq/+qpkzPJ+iq9XVqGjk7Kezo+Samu/Wlicy/6UzzodqhJZ51UJaujoHpXqvOO7rR7g05zjr5f721FD9WJ+k4EYjynu+CeK5uUQ+5W7JkawCPJxcmYdVd3wjMzM2zbtu2m1EMCyFJKpJQpUPf6Dwsh0j9JxmTiUdFoNLh+/Xq6fVcoFFKg9jzvjgwPv/nNb3LkyJEtX8P3fM/38OUvf5lcLve6n+sNqremDjlJCHnggQf69ufh9rb1po8dpLBzigv//elUF4xnQzC8YGGPFYjWGjjFHNgmrUSZYRr4q8PAEbV9MqJIfrKM3+r08bqZiVIKxqBXsI2qj7IMitPjNK8ua7+DnkoWVABa11cQpkFp1wyttSpmN214sLLjJZoXFrBtk8KuGVrrNYJaS68Y5zJ9xx1vtDCB8h6drdZS1T56RgmBUczQ6pEBFsYrWHmPMIoIl6v9ZupKEWw0yE+N05lfJ1spYeU9Ij+gvVIlPzdOvNog7p7cgu6x2IUsxUoRpMKdrSDDmLCqL6nzE2XirjY3Xm9qztgQBDmTXFug8h6qmyUnpMRwHHJLdc3Dpq9DyxetHiN5AcQlF7MZa8DL2N0hmqWHfFLpqCalOWIsA9kJCQyFshW2ZWNmXVQ70F20aeifK7oaiEOp7TK7lp6yFWoOOOtoo/y8g/IjjEpWd8/G5gKIMZnX9JNlYLUCEDHB0e20Vq4yUagwMTFBs9nk0qVLKfWQgGXyxzTNFOQGwS4B597/AqnG13VdPM9jamqK9fV1jh49StCN6KrX6ywsLNDpdLAsqw+ks9nsbZvZK6Vuep9Op0Mmc+cWXd6suucAWQjBAw88MJKasCzrtgyGslMVDv8//wfX/vF/s/qtC6jW6Pvajk2Elp4hBMWd0zTmV/Tg79rwgoidz9BaWNPSrK5VZu36MjKKMazRX7rC9Dita6t4xRxm1qUxvzkIDAbkbSqWtK6vUNg5ReiHKLutPXm75RRzKeDKMKZ1fQWEoLR9CsNzqF/sV1PApuJDhjpAszg7iTKgubhObrqSpqQkFVSb+HGI7UucQg63mNVDvWoTw7bIjZXTROmw1iKsaU10ZdccRDH2dIWw3iKqa+D3JkqITkS00e9XbOUzuLkMSgjMqSKyEyLrbaQlMF2HfKe7VlzvENc7iJyLbVrI1WpqKiRcC1xb/zKEMcqK9bqxVARFG7facxLuRIRIzCDC8HtOdAY6yXmxrreckxsrGZ3c3HvQE7n+rlptrl5vfvqx3sZb3LyvMnUYal+QasGFRgdixcK+DEsrVzl06FC6pTYxMbH5iHGcguX8/DyNRgMpZTrAS/4kXhEJAPZemveCc/Kn2WymG7KWZTE2Nsb4+Hh6/yAIaDQaqeNbq+v41/u8r+adfLMr+eTfvk19kG+r7jlAvllZloU/IpXjZmXaFru+91FK923j/H//2tC/O6VcPxgpRevaCm4xu+UXxCvnUzWGiiXNa8t4+QzORJHaCDBUhsDvytfCepuw3iY3NaYz0ly7T9mRHtdYnubVFU11miaZnRO01uuEzTaWa+MPDiCVwrRM2leWyU1UMDybxsIqKo6xMi6EsR5edqvTvQqo7JpBSYWaKNHp4aJl1sYNdNeomj6dpn7frVIWp5Albg9fqRR2drnknrLzWbzJEiJSREELxeb9nLEChi+JVvvtUAPPIJvLYHsuoggEMbLpYxUyGO0IlayRS6XDT7MOLNaRvb/0pl5/toNIb82ZGlwiJZFBpNfILYUZo+PxHLMv/RnBZnJzb03kUcsDa/g9Phjp3ScGVq8tQ1McG93PzTa6XbRBKCTnMy0qU9Mc23Z4y++daZqUy+W+oXYCqPV6naWlJc6fP08URSk/XCgUKBaLuK7e7OsFaSklFy9eZHV1lQcffFBz8SM6aYBSqZQ+b8JLJyB9/fp1ms1mX1xU0k3fjnfyvQDI9ySHHIbhSE/k1dVVlpeXOXTo0C0/Vu9wMG75TC7HtHo44fz2qVQ10FvZuXHaC2tkt03QWlwj7vKnVi6D8oMhrhk0WAVxhJAge/TI5ngBtTa8vu1WCpiuReSHm7QKaMmVZ6ertGkZgsKeGTorVfyNfgDQXHPQxzWbno0zXkQoRoJ+fvsk7YVNADXzHr6heUwnFsgBblqYBl6lkGqjpW0QuSaGQoemrg8PJTOzY8Qrm+ZNZt7Tf2wL0QqJq/0nFlVycXzVp9kFsKeK0I4wPBthdVeZgwgz7w5n3XkWwrG7CxY9xz+Z31RiJJV1CKMII5IoQxALfcpQtsARJoZlYlqWTm62tEJGz/I2KY7+Y1U6Eqo3KSQBmnYIUaz/Le+i/BAVSa7ttNl+5FAKmq+3evnher1OrVYjCAJc16VYLKbd94ULF5idnWXnzp0j6Q7oHx72Yk1CQfTSJb3eyQk3nSyE1Gq1tPMfXGoJw5AnnniCb37zm3fk9b9B9dbkkG9Wt2vBubGxwZkzZ8hkMjzyyCOcO3eObY/tpnPuBtf/6TmUUukm32CprldD6/oKVtaFvEu81iQzVhgJ4LLoEtfaeonAEOR2TNFaXAPTQNQ7I8+EnaCDuaGfx5koErY6qFaAO1UiWhrmrp1ijtblJZRU5OcmicOQ9vIGwjAwTHOIa447IaZh0J5fJT87jlRKe1QoRWaylCoo0p9vdHAcCyfjYWQtxKSNv1YjavkgIDddwV/cfL+MUOKEEndmjHi1gSy6REoi/AgzUBhjOeKVWh8Axo0OVtYjWqyllpeBhQ4yzeW0D3HUfyKwp8tpVxo3NxNNrKkC0fyGzpLLOAjbQtgmhiFQvs6hoxMiYqXBeKCLjbI2+AG2FICAGCzbxM0kOXh6AUQS0HQk+WATtBRKqypuDHxOYzno7aBNodNCetNDsg5xJ8CIJJ2Hptl3/86hz/r1VCJzy+VyzMzM6OPtehhvbGxw6dIl2u02tm2zsrKC7/spUOdyuXTwB/28dO/wMOmkky46+a/neWQyGaanp9PhYb1e5+TJk1SrVa5fv56a0efz+bS7fy1GQoP18z//8/zN3/wNjuOwb98+Pv/5z6dd/es0Frrluic75K0c31qtFmfOnOHhhx++6f2bzSZnzpxBSsmBAwfSD/uVV15hdnaWSqVCZ63Gja+/zPorwxFK3ngJf0R0kjlRwAhlqg3uLaeSJ9wYcB3zbLLbJqheWOgfigHZ2XE6N/ov7xUQFR2MIMYYpAQEOr1krf9y2R0vYhUzNC7eGPp0M1NlgpX+12FltUIkbgXDA0tDkJso4w/cxxkvYpcy+Dc2iAdc57JzEwQ3hk9qzmwZGcXEUYRoBFr1AFDKYLeioffDmSoh17QBk5FzMbIuwjQwbFsvdfQYOWEIrPH8po44eYsKns668/vfO2OqqAdwlomwdHhA028hEOTcDAYCYomSCuGYqGpHL4KA7osq2X7TeRQtD3I9GKsUqEoGo0d3jAEUMzqM1DLBNJGmohkF2IaFd3gOe//00Hv3RlVCaezatYvZ2VmEEH1DvFqtppdeDKOPk05CULeqrYaHsHkiuHjxIg8++GDfUkuj0eCf/umf+IM/+ANeeeUVDh06xMMPP8zP/uzPsm/fvtt+fV/60pd473vfi2VZ/MIv/AIA//k//2deeeUV/t2/+3ecOHHi9RgLvd0hD9arDfV83+f8+fPUajX279/P+Pj40P2TDtsbK7LrA4/hjRe58fWX+ygIMzOa9zIMgWr75LZP0ry+nAJgZrqSDrh6SxgmrUtLZEp5hGvTSgBYCOIRmmMBlAtF2jfWsWcqBNUmsuu5oYreEBiDVnG0L6/gFvPYhQytxTVkGGM4Vnrf3opaPnYhS7TeQOUdLMdGVrVhfX5ugs788Eai6dp0LumrAne8hOnZBLUmVsYlGIi3AnDGCxosY5ka/VilLGRdVBASi5DeXwVVdFMwBpBNH9n0cWYqRNf18QjHwsi52qPCsbS0zLNToBYFDxFKCAbAeLKA6nFWU0DbVuSUTmimqVeesQ2EZ6fUkup6IlPOQiPQ6owkfinnkG2FqPymDjg0FWajjTIVRlf7obI21kZb/yYHMVLEBBYUQoFxeBLzLoGx7/ucPn0aIQRHjx7tc3VzHIfx8fG+35VkuaRer/cZzQ8ODxNd8c2Gh41GgzNnzjA5OamjsrqNlmEYlMtlPvKRj7Bnzx7+8A//kKeeeooXXnjhNS+IPPHEE+nfH3/8cb7whS8Ad89YCO5RQN6K3N+KsoiiiEuXLrG4uMjevXs5fHj0YGTw/sIwmPmOByjt28blv/8G7aUNDM8eUhx0DwpZ76DCmM78KnYxi2nbdFaqwz7G3fLG8rSurxJ2VRS56QpxGCFcG3/Ec7jjxZTTDZf12nZ++yRRxydq+gyy1gqIQr0qHdW154bpWGRnxhGG6OPK02OareAvahC1WhG0IizHIrt7nKg+wnO5UtCyt26Fa3VC0AM2KfBmKkg/JFjTvr1WPgPtcJhjVwrWmqlaRBQ8zJyHFAqaAVJKjJ7PTJYzxD20jeoO4uzJItH1nvfONDDHc4ABGYHA1TxvECFyXh8Yg7a6zMb0RCqhvS6yCU2x+eaKUhY1EHLaZ4tJNx5vMo+V0iH6NURlF3tj8/1UKDqOIOtDPFPAeHB26L2+05VsuV65coX77ruvL93jZmVZ1kij+UajQa1W48aNG5w9exYpZTrESyiPBOyFEFy5coXFxUUOHTpEqVQa6qSTv//93/89169fp1Qq8e53v/uOvPY/+IM/4Id/+IeBu2csBPcoIG9VCSeVlJSSa9eucfXqVbZv3/6q/hZbAXpmqszB//sJbvx/L9O8vkzjyjCQGeWMNohJnrveQdKhsGdmJICbWXfodn9ZB59GRQc76xK3+gHQMAeOXSraC2vktk1gex5hq03QQ4tk58bxF/qfQwYR7XoTs+HjTZUQQtBe2tDonXXwl6pD117CMvFvVJGdEDufxS5liZptzR3H8RC4CsvUPh29xkeOhT1exLANopV+SsdwLeIgwgz144ju+ycQiFCb/AjbxChmELbemjRGnBzikos58NjCsbQRfXtgOWa6SHxjAyyD0FBIQ+AVc9hGV0es0IsjkcTIWKhqR8vouidXMVkYVk6MUlOM4KZFH0DrUmM5Must/JzJpYpP49lnU2qgWCxSLBZv2XfiVqrZbHLq1Cny+TzHjx9/3Vt4hmGkx5mUUirlgFdXV7l48WIa0dRqtSiXy7zjHe9I9cWDnfTS0hI/93M/h2EYfPrTn76l47gVY6FPfOITWJbFv//3//51vebXUm8pQE5KKZXyYZOTkzz22GO39IW7mWxOmAaz73oH7aV1rvy/JzSI9ZQMY0b+qgQxhhB42yb0QkhCY4wVtT54oMzxvA7bNAzyO6ZoL28QdwIy02X8EdtrTimnu+bEsnN2jDiICVptwtVhCkOYBpZUSCBMQjkdi9ACxzYRg8oNwM3n9MkCiJsd4q5CJLdjEqUUoWEQ1TY5VG+yTLA48P4EEYZSBNd0h2+VshgZl6DVIohivE4/qBuOhdG9H2gjoGi1gVnJoRoBxBLpmHrrzrFQpoHZaBMbOoUJIDZAoBADfLs5VUQmnXEksQHhmXpbcgC4jYk88Y1+hYsxWUBW24iMpRUT3UUP5UdQ9lLVhHBMVBDr2KXkg3dtZCsgcg3iKMI2LJ1gstwA2yT33vt5sGsu30sNXL58mUaj0bf8USwWX5W/HSwpJZcvX2ZpaSntTN+oSrTI+Xye2dnZVEa3vLzMrl27CMOQ06dP0+l0cByHYrHIysoK2WyW06dP85u/+Zt8/OMf5yMf+cgtS95ezVjoD//wD/nbv/1bvvzlL6ePebeMheAeHeolzm6j6qtf/Squ6w6tVd9KLS0tUa1W2b9//01/TknJta+9yOqzZ7Q/QzGLrA8bD5kZFxmE6YDKruRB6KUKwzBSsElKCAMr5xH1SOIM28KZLCKDqM+TIqnsFvx0fvc0UTsY6sJzOyboXF8d+nlrqohcaRLnbeIowmhF+nJ7ojg0HAMtV+sFXauUxcy5CMPAvz7MM2e2jREsDB+nqmSwagFWOYvhORp4qy3ccm5If2xkHO0f0RlQWUwVkT2Jz9q/2AHHRAYRMowQQYwZQpQx8XzVfxVgCozEUa33+aaKQ5piMZ6HtWb/b0rB04b0PVcKIufo1eoebTc5B+XHerCY/FzR06GmUmG95wDG7M0BMtH31mq1FKxhk79NQHpUA1KtVjl9+jQTExPs3r37jnXbt1KJkmJycpJdu3YNPXfiTPfnf/7n/PEf/zFXr17lwIEDHD9+nF/+5V++ZTrlZvXFL36Rj33sY3zlK1/pe7yXX36ZH/mRH0mHeu973/s4e/bs20O911PJcCAMQx5++OHXFIhomuaryubCMOT8+fOs2232/ODj1J45ixDQHgHIouDC8ubgLLmEz++ZSR3leiszO9an+wV0FFIskY0OuW2TtG6sphSBN1EcCcZOKUfrih4qepU8RtalvbCOmXNHctNm1tN0i1KY9UB7/OYyqIKDbAwP/oRnEw4MEKNq1+cjiLAKHlYh2+WO69jlPMHi8MnEniwiu6AbrTcBDfzu3BhxK8CaLoFSeojXDjEzLvHAJp9Zygxl5ql2gFXMEi/WMCC9chE5B6KIpqUwHItYauWE4Zi4YYDpmBhhrM3zJwrDYJx3odruB2PHhDju55xNobvkXjA2BGHYnzqdapRjhfHA7KuCMejvaKlU6utsE/42WWeu1+tIKfsc3JaXl2k2mzzwwAN31Q9CSsmFCxdYX1/vi04bLNu2+cd//Ef+6I/+iF/7tV/jwx/+MLVajeeff37L+9xuffSjH8X3fb7ne74H0IO9//bf/ttdMxaCe7RDVkoRBBooOp0O586do9lscuDAAS5cuMCDDz74mkT01WqVq1ev8uCDDw79m5SSK1eucP36dXbv3s3srB66xHHM2osXWH765f6OVwjNXwbD8jRcCxXEuJMlguVqCrBOOU9Y7QccAG+8mLqgmRkHu5KntbCGN14c2TVnppxQRokAACAASURBVCv4A+oGw3Pwpsu0Flb6TXGAzOz4SKDOzI7hL27gjBcQtkVnqYqKYoxyFlEbVoE44/kusPY8b9bBqRRRcUywWk+7QzPrImKV6rmTsieKxKvDskF325gOEbBMHVvVCiCKMV0bNcC1W1Ml5ACYKkMQmZtBounPTpc26YtutR2dQG1YJoZt6SUVy0IYaOUFaH5ZKYTbVXJ0OWdQiKwLjc1jiuKIhozImw6WbaVqDOHZEMSIvIv1+F69XHKHKlnCmJ+fZ35+Htu2MQwjBenBIdsbUdVqlVOnTjE9Pc2uXbu2Doq4cYOf/dmfpVgs8tRTTw2pn/6F1C19ePcsILdarZSP2rdvH1NTUwgheOGFF9i3b99rOqs2m03Onj3bp2NWSrG4uMiFCxeYnp5OL/WSDaXEIStstFn4n89TPzcPaDDrjAC5MGvitHsd5izMnItpmAQjOGJvqkywPAy67kwFwzRo31jrG6q5E6WR3LFRzOjNNEPgTpWJ/YBgrY47XiRcGwGAU2XCAb2xsAwy2yaQ7UDz2b3frbFMX5JyUs50hah7clCAXclheA6GaRLMr/V9iw3PRmCgBjYAnani0CBQP3YZYqkN5BUoP9S/9M1++gAgLNg4jQEueSyvEz56v/W2geHYXXvPzWq5kBsYLxjTxSGVhjFi2NfOGWRacvTP2SbO9z2Ikb+zoaRBEHDmzBniOObgwYN4nte3oZdQHkEQkMlk+kD6dmi+URXHcSovPXz48JYduZSSP//zP+dTn/oUn/jEJ/iBH/iBf8nr0W9dyiKKIk6cOMH27dt5/PHH+/io293W663B+66vr3PmzBny+Xyqz5RSpj+TgDFoQ6GdP/Ad1M7Pc+N/Pj9ydRqgkMvjtzeBTvgR0o+Iyh4ya2G2BnjlLY5VSEVncQ2r2zF3Fte1edEWX2ilpH4sqdJu2B0rYuczm3RDUoYYAsXk9QbLVWQ70KqJsQIyiolaAaIRDp3NZc4h7FFtCDQ14czYBNfXELaJVc5hWCZx28d0bKKVAYDLOsSjuvGZst7m6/thgZX3ULaBKLh0Qi2XyxXzuI0gMdjUx+J1PZUHDtos55AD4G9OFcgNAG/HFXhLtT43OXLO0Aq89CwywYAKJetou03APrrzjoJx4oZ46dIl9u7dy/T0ppZ5qw29TqdDrVZLrxB9309tNntB+lbAcmNjg1OnTjE3N8f+/fu3vM/CwgI/8zM/w9jYGF/5yldSE/17ve7JDhmg3W6PHEqcOXOGSqXymoYAcRzz7LPP8tBDD3H69GmklBw8eJBcLpc6X0E/EI8qGUYsfeMka//7XF8XaRezqbtZbzmlHFFN325PFAiaHc2bZiyszvBGojNW6JOUgfY/9mYrdObXhoaFzkRRG90PlDc7hn9jA8O1cMaL2n2t1iKzbVguB5CZG8dfGB7YebPjWqoWxdpKUymUECjHxBoAI+VZGIFEDHzT7OkSstrGKmYQlon0Qz3cGy8MD/cKHrSjoS7YmSkNgbSR9zSlEEtNI2VshGthZrt6ZND/jWLIOKilet9JUBQ2h26bB2sgbEt7TySvCwgdgdvTWCtAlLx+/TJglLOojTbGXBnnuw8MvZ+vtdrtNqdOncJ1Xfbv339bxj29lWzPJZ10rVbrU0IkKo9sdtNgK45jzp49S7PZ5P7779/SKlNKyZ/+6Z/y27/92/zGb/wG3//93/8vuSvurbduhwykzlOD9Xo65CiKaLVavPjii+zfv5+xsbEtO+KblR+FLJUU8ugchavN1N/BLowGZCuXSQE5kaJlt41rjfAIRUQr0NFIvSX9ENnUqdfGWI6o1sLoDpDEiMUUIQziRqd73yjdwPNmKtpT2BSoePN+ViGDvzgM0s5UmaCHmhGWge8I3Hy2L54oKS+fJRwAWOkYhKt1DNkjxUNrhWUo9XBPKmSrg2yFmJZFHPdTCtZEYbhjFgLDMpAJcCuFagWYhYz2uOj90VzX5lJKhGMhHBNsS88BXGtzMqi62mY/RmRtVNfoXrgWbDTp2FIDoVL4toHZ7qBMqb2PTQOyLnYkMcdz2I/uHnp/Xksppbh69Srz8/McPHjwdSdrCCFSM/rexiYB6SRZpN1uY1kWtm1Tq9WYm5vjkUce2VK9MT8/z0//9E8zPT3NV77ylTuSAPIvre7ZDjkIgpGAfOXKFYQQ7Nix45YfK9Fmzs/PE4Zhug2UbAzdKhBHUcTFixdZW1vjvvvuS4cTGyevsPT0S6hIDjmkGbapedPBxI+MA7HE7dpeJvezChmtAx546bFnYfv9U31vuowyTfwRns2ZuXE6o7rgbWP48+sIR3fNcccnXG/gTZeH16CFgZ33UmBPnzrnak2uUthjBYRlEjfaWPkM4Qi1hTtdHkoIkbaBIdWmx0W3rNmyVkLYOoFZtkNkJ0AZIl0sScqeKRMPPJ/Iuzpiqbe7Fl2qYlDBMVNCDoC8OVEYojTirI3ZDvvoC9E1LqLnpCZdExFJDAnXJhT1CSddpnitqdD1ep1Tp05RqVTYs2fPG5p7N1hRFHHq1CmazSbj4+O0Wq3U6yJ5TbVajd27d/OFL3yBz3zmM/yn//Sf+L7v+757pSvurbd2hyyEGAnItm3Tbo8IIx1RCd+W2Aw+/vjjfOMb30j36XtdrW5WUkquX7/OtWvX2LFjB48++mjfF658eCeFvbOsPHOStRcu9F3+epMVOqNogLEi7flVOgtrCMciu32C9uI6TjFLuzEiW88S0Dt0korOwjoq52CWsximkV76C8MYqeYwerwnVBCl9ERmbhwhwMh6yB6PDW+2QjAC1O1chqClHycZDArbRDk2zmwZFUlNuUQSZ7I4Mq7JqxSIBm6PXRMWqxgDH3sra5A1bIxyTic2d5Uc0XJt6LfEcGxkc6C7nir1rWEDiGJmSH0hbBM1YNmp0Lp0MfBMwrFQg8+Tz6DWmhgTefY/8QBBEKTDtYWFBdrtdh8tUCwWtwTpOI65ePEi6+vrfYb1d6tWVlY4e/Ysu3bt4oEHHug7xt6Flk9+8pP8r//1v+h0OnzgAx/g2rVrd/U4v93qngXkrepWKYtkYFcoFDh+/Di2bSOlpFKp8Mwzz+C6LqVSKe1gtpLRrayscP78+TRodauNQNO1mX73Q5Qf2M3iV15Mc/jiEQY/CEHQwxGrQFMKVsHTdIJl9nXU0jGxm8Ov2ZkoEq7WUYTEgPRMpG1gWhZsDG8kupXcSO5YJV4UaGc3w7EI6i2CEZI7Z6I4WhUyUep3fTME1kQBw7OxxnJaLtcFWntiGIwBMoXsEJ/suwbZNghCoubm1YdVzuqrm5yL8GyEaWhaoR0iCh6yHSAiicg6xIMqEwGGEP2G9oBRzg3J6RjPYa/1n9y0gmJgODlRQK3q8FTn0T0IIXBdl8nJyT5aIAHpWq3G4uIirVYL27b7OulOp8PZs2eZm5vj2LFjd7XbDMOQM2fOEEURR44cGfl7YVkWpVKJv/7rv+Zb3/oWn/vc5/iu7/ouvvWtb3HmzJl7sTu+5bpnKYutTOo3NjaYn5/n/vvvH3m/xHpTKcWBAwdGDuxA82XJ5Dkx8M5msxSLRUqlEoZhcOHCBRzHue2NQID6hQXWX7hA++owneBNV0YmXGfmxunMr2F4NnHWQq43EVJ7Vozqst3J0pC9JkJgFTLaPL7RSWOKlGNixmrI9tKdKo3sYIOCjS1M3KynZXPdJQi7kicatBnNe6h2OPzYcxXC7vaesAzMUg7hWhgI/Zw9tII9VSIePA4BZiGLHOhao6KDUx9QiTim7mJ7lzVsE2u8AGGk9c0JUNhmunmnwhgVxno4uNF/5SUyts7M66U/bANhGP0Wn6bQeuV2iHVgGuf4nqH382aVgPTGxgYLCwuEYUg+n+9rGHoHbG9UJXYEe/bsYXp6esvnu3btGk8++SS7d+/mv/yX//KGrmd/G9XblMWo2sqCMwgCzp8/T7Va5cCBA1QqlZsO7JKhRpJgnWg4V1dXOXXqFJ1OB9d1cRyHpaWltHu5VQ6vsHeW/K5p1l+4wOqzZ/q55S1OoslAUHZCRCfE9jzs8TzBCAWFXc4NgzF6aJd0qpbQQzmpFEoo4sHuDx3YOljSErhtCXFEWPfBEDiTRYysO+RhATrlOhy4fDcyDlHP86lIEq3WcWYrekgoBFYpqy0vpdLbcIOvcXpY+iZcC9dXQ51F6Jo4jYHvRc4Z5pg9C6Ku6iK9EVQUEwuJNASW62BYBiLrauMjITb9K1yrB/QFoDYXhEoZrIe2D72OVyvHcYjjmOXlZe677z5mZmYIwzBVQSwtLaWddK9ULTGTf70VBAGnT59GKTVkz9lbUkr+6I/+iM997nP81//6X3n/+9//lu6GR9U9C8hb1WDydBzHXLlyhfn5efbs2cPBgwfT229nYCelZGlpicXFxXQRJXGzqlarzM/Pp74ChUIh7V5u5tAlTIOxI/dROryTlROn2PjWJcyMO3L7Lkm+7jumTqDN00FL1ZarqeTNyrpEI3jiPg9kBcFyFcOzMR0be9sYwVoj/RmzktOLEwNllfP68jt9UEWwUsfORxBIHQtlm4T1FqZrE45IN7HLOcIB43rhWpvyPKXSTtuaLhKtNpCmwC7ltJezUshO1BdlnzzuYJqKWc4hBt4LJQSqHTB4+lQZe0gdEpc8RLWj17ClgsiHSo54YcTAcLXZd0LVXXQEscI5tgfDvT0pWqfT4fTp05im2QeGo3yKwzBMOenl5WVarRamafZx0rcD0slS1MWLF9Pv/FZ15coVnnzySfbt28fXvva1u8Zpdzod3v3ud+P7PlEU8W/+zb/h4x//+F157tdS9yxlEcfxSK440RI/9thjLCwscPHiRebm5tJcsNsF4l6h/Vb5YoPPn3Qu1WqVZrOZ/lIkdMdWg5pgo8H6ixepvnBx+HHzDtZgh0e/tlnYJu5kkajZIaprCVdvuZOlPllZUl6vvljobhcFKo6HTgLCtVCRHJLSGRMFDUYD5c5pwb/0Q521p7oURisYuhJwZitDIL2V2sKZKRMuVhGWiVHwMBxbe0P4MbLl9yWImJVhBYU1UyIe6K5lzsZohn3XnpEAwxQ67LT39RYyQwM+czyvjfR7b5ssIFfqiIJH9gce1nTGLZRSKh0UjwpTuNXq7aSTxA/TNPtsPbPZ7Eizn1OnTmGaJgcPHtxS0yyl5POf/zy/93u/x2/+5m/yvve97652xUlTlM/nCcOQd73rXXz605/u8ze+S/XWpiy2KsMwCIKAZ555hlKp1Dewu1098cbGBmfPnqVQKNz0Uq23RiX/Jp1L7+VlEiiZ/PE8D6ecZ/rd76C4fxvLT7+S8sLCMbVZ/EC5k0WCHoDV5vjrZLaPY2c9wka7T/c86jUL0+g3ClKKYKmKNZYnrLeRRQcXk7irk3bHi8ODPwGy0RmyHxWlTJ8KQ9gmdiWv6QrqyOYmHWLkXMIR6SIi7w6tZBt5L+26VRQTrzeJ0c50cbfDFraJkXMxCxmIJOZkARXGyI427I9HnDxs20bSf9JTBQdzgI+Oii724LLHWG4IjEXRS13o3Id33jIYN5tNTp48SbFY5Pjx469LymbbNmNjY32bcL0gffHiRZrNZp9Uzfd9bty4wYEDB5iYmNjysS9fvsxHP/pRDh06xNNPP33HTIBupxKLT9CvKwzDb2ua5J4F5FFverPZ5PTp0wRBwNGjR8lms68JiNvtNmfPniWOY+6///7X7Y5l2/bQ5aXv++nA8Nq1a/i+TyaToVQq6UvFbSaTczuxLq1jus7I6CQx6hdVQLTeIG75KBTedBkZSWQYjfTKcKfLI5UV7U4HJ5IYdUlMiFXOYeZc3XkPPUZlpL44juP+KKYwJmx1oHscScK0kgrDMghb/d2xkXOhOsxhm1mXqNl/uzmWT8E4ea642oZQ6o65p6yZMqreQeQdhGVqUx/bBD9CFlzCdgcbA9OxcQbVK4bAHPCMVkDkD9Mfhm0i0WBt7nz1DldKyaVLl1hZWeHgwYNv2DBsFEhHUcTq6irnz59HSolpmly4cIHl5eUhg3wpJb//+7/P5z//eT71qU/x3d/93W8qCMZxzNGjRzl37hw/+ZM/yWOPPfamHcur1T0LyL0VBAHnzp2jVqtx8OBBOp0OnufdNhCHYcilS5eGFjveiHJdl6mpqb6h4draWnoisCyLZdkm/+A4uYbCrLeJe7pdw7NT0/je8qZKqZZYINK/Z3ZNInMZrd7ooRsGlzoAyNo4A6buUbWFlfOQzQ7OhOaIg9U6Koy089pA2ePDnDdAZGx+KeNGh7jRQeQcRDtEehaBIcl4HsKPsfIZooHHNkvZkZI4MUqTPl0iGhza5VytOZZKr0TrNwoj56G6/68vziWi4iIjCZ4OPxWmHuThR3qGJwQKhRKakglyxmYmnG1iRyFWJYPzzh2v+v1LnNGmpqY4duzYXfUqTmi5a9eucfDgwfR7P2iQ//TTT/O5z30OKSXbtm3jM5/5DMePH3/TO1LTNHn++efZ2NjgIx/5CC+99NJIx8Zvh7qnATmOYy5fvszCwgJ79+7l0KFDgFZavPzyy6lv7KsNMnoXO3bu3Ml9991317WdFy5coFqtcujQoZTukFLSbDY13VGy6VxcInO9iRlIHR0/4rJ71BRAmAb+wgYqjLR9ZzlPsN7Qm3OjlBilAn67v2sWhkg54NRNzhBkdkygQon0A52Okf78MKCYpSyiNry0EwqFo8DoRHiAarW0AqLexposgqkd4OJ6Ww/0Bu5vTRaJB7lx0xjyTgbt/RwPKD7ioouoDcRlFTz9mAoIIv22WgZGw0f12axqCsVs+P0dsm0hNnz8nMnJa2cR1/WlddJt5vN5DMMgiqLUPvbBBx+8q17FoK8GT548STab5dixY306+t7svDiOefrpp8lms/yH//AfkFLyu7/7u1y9epV/+2//7V095q2qXC7zr/7Vv+KLX/zity0g37NDvU6nw9e//nXm5uZSr9VkYJek2SaUQKPRSAdrCUgnuuFksSNJUXi92WK3U8ng5urVq33R6zerKAhZeeEczTMLqWNYUsKzEEE09ClqE6F+gFVCkNk+QXu9hugxoReOhZBqyK3Om630L3Z0K1V/GCJdk5ZRNNLTOBnE9VbsGliBGpqIRAUbe8AuUxQ9DCW0MZAQqFCDtJXzND3Re1wz5aHu2Ch4qIbf9/4o0dVgD5ggjQJ5a7o4NAg0J/vTSgCMSi79bLLvfwBrrtyX9JF8J6MoIggCJicn2blzZwrSd6OUUly7do3r16+/qv/FhQsXePLJJ3nnO9/JJz7xibt+0rhZLS8vY9s25XKZdrvNE088wS/8wi/wwQ9+8G4fylvXDxk2dcGmad6SE1vvYC1RP4RhiOu67Nq1i4mJiTfUrHuwEnpibGyMPXv23PaJQMWS2smrrP/zuU2VxXgWsTZKppYjrg6Ct43q6Jgms5TFzLoEK3XcyRFDOxJHugGlQiVPvD7cpTszZVQQYXgOstkhqrUxsq6OXhrcfhvPweCm2xYqjChnD20kGmM5hB9iZF293KEUsR8iYjm0umxPFvu0zwBiIgerA89fHFZQCMsYTgEZoDqSMis55HoLc7JA7vseGnp/El2vlJK5uTna7XYK0m9ksGlSrVaLV155hWKxyL59+7YcGsZxzOc+9zn+5E/+hKeeeuqOJT7fyXrxxRf5sR/7MeI4RkrJD/3QD/Erv/Irb8ahvLUBOQHWcrmcgvAtObH5PufPn6fZbLJ7926klOljhWFILpdLu+jbWfS41Wq1Wpw9exaA/fv3k81mX9fjqVhSP32NtX8+h5ByiBNWWRujPUIeWPKwBgdmpoE3WyFuB32m9c4WfLA7NWw4NAp4jayLM1VEtgKt6Oga7hie3Y0wGrTRHO6kzWIGOcIXOcqY2AMWpeZUEblcR3g2RtbFsE0wdVxS1PaJWz6W1MscRtYdkThSHFqSGd0dDxsNJdaaAJn3Hsbevjk4U0qxsLDA5cuXt9T19some0G6V5EzSqZ2K6WU4sqVK9y4ceNVA07PnTvHk08+ydGjR/n1X//11/09vdW6evUqP/qjP8ri4iJCCH78x3+cn/7pn74rz/06660NyCdOnODnfu7nUt716NGjHD9+nHe+850jvViTBZHFxUX27NmTJoz0VqJpTAC6Xq+jlEoXPW6Fj96qep3gEmvPO1lSSlrnb7Dxz+e0J3G3vBEexikfGvV/3PZkIU3mMAsZzHyGcKOBU8wOA29mdMfrzo0NGQ4Ju3tSiyRK6I7c8lyEYxBeW+9fpNgCpO2p0tDChzWeH/KhUAKUZWAOvDYxnu/n3E0De7qE8iPd/RpCa6+FQEQ6nFSGsU4hMQSGafZzx3RPEqO0yKtNjHKW3A88nH5XWq0Wp06dIpPJsH///tu6IkqGawlIJ9r2wU76Zt/LRqPByZMnqVQq7N27d0tAj+OY3/md3+HP/uzP+K3f+i3e9a533fJx3olaWFhgYWGBI0eOUK/XOXr0KH/1V3+1pRXCt1G9tQE5qTAMefnll/nGN77Bs88+y/PPP49hGDzyyCMcOXKEI0eO8LWvfY3p6WmOHDnCjh07bqu7GLXoYVlWykcnGuKtfhl6u6Lt27ezbdu2N5wnbF1eYuO583Tm1zBde8jyc1RnCxDmLJxWf7cpMg52IaMd4lZqKbfszo4AXrNrfDQAXGosizGw/SZMkVIMVjmHME1kJ8DMuMMbfFlXm8EPfJetifwQV22PMKmPLIEZqXSrMamUU+59zOnikPObOVNGbrT08Vraq0JkHUjWphEgFJiGztwT4Bycw9k9kWYxJl1prz799VQURSlA9y58DHbSSikuX77M8vIyhw4dumn475kzZ/ipn/opHn30UX7t135tS5P5u1kf/vCH+ehHP5oGk34b19uAPKqSgd4///M/82d/9md84QtfYPv27YyPj3PkyBGOHj3Ko48+elNzlFerMAzTgWG1Wk1ldkkXXSwWsW2bjY0Nzpw5Q6lUYu/eva85weG11srF6yw/cxJ33e9L6BhJNWQcDaQDG3h+3sJrdkHaFJhjef0zrbB/DRtwZ8pD6dIKwLUwBoJVNS0xALymQHguVsZB2BYqjIirLeyJIuGNAQqjlEEOKjYEmPkMcoC2iUoudrX/WKP8MB9N16+4/z0QOnJpUH43lh9Ku7YmC8TLdUTWofh/Pkq9ob2Kx8fH2bNnzxt+Ih7cyqvX6/i+Tz6fZ/v27ZRKpZEmRFEU8dnPfpa/+Iu/4Ld/+7f5zu/8zjf0OG+1Ll26xLvf/W5eeuml15Qif5frbUC+Wfm+z0/8xE/wi7/4ixw4cICFhQVOnDiRdtJLS0vcd999HD16lGPHjvHII4+Qz+dfE0gP5pJtbGykl5Vzc3NMTEy8IXz0VuX7PufOncP3fQ4cOICrDGovXqJx8irKEDp1esh5bbjjRWhjoHhgaOVnTeyu9M6UQoenKh1gGg2avI9YJwawipk+XTXo0NJB7ljYJmbBw3AcHRPVCYhrHeypYhqemj7mKPmbZSAw+s2CAGtseCAZFu1hl7ixLKz1H6dRyqAG+GydIK1PaO4ju7iW8dnY2ODw4cN3fYNNSsnFixdZXV3lwIEDxHGcgnSS8uE4Ds888wzbt2/nk5/8JO9617v4+Mc//roDTu9UNRoN3vOe9/BLv/RL/Ot//a/f7MO5lXobkF9PxXHM6dOneeaZZ3jmmWd47rnnCMOQhx56KAXp+++//7a62jiOuXTpEsvLy+zdu5dMJtPHRwshhoyH7qTeWUqZRvns27ePycnJvseXYUTz3ALV5y4MWWRapWy6Hp2UM1kc6X0xeHtsCXxP4EgDqx33deOjHsOeKIzM+LOKI45htjzUHYusi+laGI6NECCDCNnoYBYyQyBrz5SJBrvrchY5IJPD1LFVfSoKIPKGcw3FWA4GJIfJ0E+ZBmf3GMzu3M727dvv+tJErVbj5MmTTE9Pb+m7EgQBV69e5Zd/+Zd54YUXsG2bHTt28MM//MP8+I//+F093lEVhiEf/OAH+d7v/V4+9rGPvdmHc6v1NiDf6Wq1Wjz33HOcOHGCEydO8Morr6Q+FseOHeP48eNs37596Eve64o1Nze3JU+d8NHVapVqtUqr1UrNvBNO+rV2KImMLtFT36wbV0rRubpC7aXLtC8vbSlfc2cqQ3aaRs4dunyHro/ESh1p6rRpGcc6ESQY/qY6U8MpIdZ4YaR22SwMUxAjQbrgEbd9pGuSyWcRwkAGEYZiaEHEmh5OCNG3DQN3POCBLB0DI+zXTSuhZXEilNTGbabe//Bd7zTjOE6Xiw4fPnxTrfDJkyd58sknec973sOv/uqv4nkey8vLrK2tpW6Ib1YppfixH/sxxsbGeOqpp97UY7nNehuQ3+hSSrG6usqJEyd45plnOHHiBFevXmXnzp0cP36co0ePEscxzz//PB/60IfYt2/fbWuZE/PxhJPudDqpp0UynLlZl574biSG+7c7iInqbZrnFqi/eIm4R/4lbBOhGFoQGUVtKEN/F4cc2eYqxJ0AKUA2fIwgJjLBUgylTjuTw0b4upMeTPMQGJ6tFR491ckaZFoDyozJItFKHUyBkes6wtkGhhKoWKLCCNWJUEGImXP7zI5AUyCDuuVRLnF+zkx59kt7bbzJ8i2rH+5EbWxscOrUqbQZ2Or5oiji05/+NH/913/NZz/7WY4fP/6GHtdrqa997Wt813d9F+94xzvSpuY3fuM3+MAHPvAmH9mr1tuA/GaUlJILFy7wpS99ic985jNsbGywc+dOdu/ezbFjxzh27BgPPfTQlpFPr1YJH5100bVajTiO+xIiCoVCOj1fWlp6XfaM6fPGktalRRonr9G5tqK74xGbeVbO6wNuAMayiLURAzbP1u5q3TILGR3XVO8QV1uphWdsCkw5DNIpoPbeNj0sf1Om0AqKgZOHNV4YksWNHYNOOwAAF3NJREFUojCM8Ty0A72laJsI00CZAiGFVnZICbFCRhGGZaHaASrc5OEjz8DuSKyZEt777r+p+uFm9qu3W3Ecc+7cORqNBocPH76pVviVV17hySef5L3vfS+/8iu/8pq/n7db//E//kf+9m//lqmpKV566aW78pxvUr0NyG9mfepTn2L37t384A/+IGEY8uKLL6Z89Le+9S0cx+GRRx5JQfq+++57zVP2xNMiAen19XV836dQKLBt2zbK5fIdjfCJ6m1a529Qf/lK3+BtZMdKN7ZpkLudKhGNMD8yc57uRA2BVc5pw3bLILzer0eOLIEVD3/L7bERzzVTJho4eRiFDGrIOElgZGwdJ9V7TJOFoWHgKODWCyObJwMFUHQxOxJMQfY79uPsHLar7N0STUA6CTPttV+9nc9vbW2NM2fOpFLKre4bhiFPPfUUf/d3f8dnP/tZjh07dsvPcSfqq1/9Kvl8nh/90R99G5B5G5DflFJKUavVePbZZ1Oq4/z588zOzqZ89LFjx4aGbq9WSR6gbdvs3bu3z8Kz2WziOE4fH/16uyClFP78Gs3T12ldXMQeKwx5Fpt5D9kYtsgcxRPb46MB3cx7yI62+BS21jELxxryoggdgTPs0d89hn7wHeVlYU0Ob+CJjIPyg4Fv/mjgtsZyQ1x7klgtMg6l/+sxbeV5C5VkNiZ/EulkL0iP+vyiKOLs2bO0220OHz58U4rqpZde4qd+6qd44okn+KVf+qW71hUP1qVLl/jgBz/4NiDzNiB/21Ri5vKNb3wjHRqura1x4MCBFKAffvjhLXWiSeT7gQMHtlwuGAxm9X2/L5i1WCy+JvMkpRRLCzdYeP4M4x0LVjZjitzZCsHCgJ44kYANJoJMlYcAXW/bDQwURVdJ4Vh6vVoqokaHjiE3NdHdCjMmTnswzkNg2Gaf+5x+ruHBoTUzbNFpTRSIB1aiRd6D5oDUzTZBArHEe2gHmSO3F17aW0qpPpCuVqsEQUAmk0kBOhncvZoRVRiGfPKTn+SLX/wiv/M7v8ORI0de83HdiXobkHt+6G1A/vatKIo4efJkqo1+7rnnUErxzne+k2PHjnHkyBG+/vWvMzY2xuOPP37bMiqlFO12OwXohI9OVm5LpdKrOoy1Wi1Onz6N4zjs379fB262A9oXbtA6f4O41iEeACp3tjzEP/eCV28506NAukC01t/JKrTCw8q4GLaFirXxvMjYQ34SQd7CHQBukXVhUB0idObdUCc8ArjbWcH/3965B0V5n3v88+4uICzLiuCdKndDIYqXNaZpZmwSjXdjPBMT9SQZk7RpqtVktKYxWk2GWJvexpiOIWmKpjH2xFOHYydD0jrGmgsXJVSJoEYuihdEBFlg7+97/oB9ZXcRURZ2xd9nhhkXd97fA+w++7zP7/l9vxEWr5OCQ69t8EUtnIzW4N/JCvffr6Ghgerqaux2O6GhoR4ynt4fsseOHePnP/85M2fO5JVXXulTwazrIRJyhyeJhHz74NbSOHLkCLm5uXz44YcMHTqU2NhYMjMz1dG7ESNG3HK/WJZlD2lSs9nc6aZTR/cKt0t3Z7harFgqL9F6+mLbiJwCIQMjfAxWQ4d1Mqqmax/N89atGGr0SdKdKrW1b8BpI8LaNuRom7VWJAnFq+p2GEIINXtpUcQaPFxGoK2FgdXhI9GpCdH6zCi7BY90wwdieNhX1c0f1NXV8d133xEfH8+wYcOAtg/Jjn+/iooKdu/eTVhYGKdPn+a9997j3nvv7ZV4bgWRkK/RrwXq+xtuf7D777+f999/n08//ZRx48Zx6dIldcMwJyeHCxcukJCQoAoqjR8/nqioqG4l6Y7KYW7cughXr17l0qVLNDU14XA4MBqNJCQkdDnTqtUPIDJjFJEZo3BZbFirL2OprMXZbPVItN7tA2ivhL01NUK01wTwO9JJqaCLMeCsvYrTdi3RavRhKC22Nk+9yAGqoH2ooiDLNrA5VVElq82K90ChzhiB0+L1YRDru+mnMYYjX227MwhLGeYbXA+x2+2cPHkSWZZ9/Bz1ej16vZ4RI0YAba8bi8WCwWDgvvvu48UXX2TWrFmBkqEUdIGokPshsixz6tQptR9dXFyM1WolIyNDTdLp6ek3fbtqsVg4efIkkiQxevRoj5603W5Hr9erVbTBYOiyHy3bndhqLmOprsNRb/btE3OdqY1BET6n4KQBIWBz+YoLRUfiarzxxIXP+JtWQooKR5EVXMg4ZReyS0ajgDYkBI3FgUa+NlfdmYiRu10hheowPjalTTHOT9TW1lJRUUFiYiJDhw697vNsNhtvvvkmBw4c4J133mHs2GtVuttZPdA88cQTfP7551y+fJmhQ4eyadMmnnnmmUCH1RuIloXgGjabjZKSErUfXVpaSkREBBMmTFA3DePj4zvtF8uyTHV1NbW1tdedaXYbAnTsRyuKos5Hu6VJO7u+oig46pqwnrmMraYeR91VpAGhYHP4vOKcep2Pw3bosGgfBThNpK8wPIA2ItTH46+z2eWQYUafFoqsD0Xbcq2fLEvgCJPQaXRoddq2TUatFkUDGkkDSltijpgQ7xPHrWCz2SgvL0er1ZKamtrlB2pJSQkrV67kkUce4Re/+EWfC1cJfBAJuSPr168nNzcXjUbDkCFDyMnJUW/p7kQURaGhoYGioiI1SVdVVREXF6cm6IkTJ/Lll1+i0WhIT09n9OjRNzUrLcuyhzTp9ayyvCs12ebAfrER25nL2M5fuaZfMSCkLUl7oTP6Op50dnxaO0jvI2TUVrn6HhrRGsN9bJ+chlBCvMSFNEMMyHVem4Z6LQPaZUr1D48lbPj17Y+6g9tktKqqiuTkZAYPHnzd59psNrZs2cKhQ4fYvn07d999d4/Wvlny8vJYuXIlLpeLZ599lpdffrlP1w9iRELuSFNTk9oX3bp1K8ePH2f79u0Bjiq4cFfCBQUF7N+/n9zcXKKjo9WpDpPJxNixY3ukg+ttlWWxWNT5Wvfonbvycx9uGDYwhsFSm7CQtbLWQ2BIE+mrWQzts8deFXLoUKOPWlxn1bEmqhPpTo2EpPXduNMYfZ1KNO1tFWeYhqqkMPVOobuTKx2xWq2Ul5erUyxdVbrFxcWsWrWKhQsXsnr16j6vil0uF6mpqfzzn/8kLi4Ok8nERx99dDuIx/cFYlOvIx03qVpaWoKifxZsaDQaEhISiI+PZ9euXeTk5DBt2jRKS0vJz89n586dHD16FK1Wqwr8m0wmUlJSui0dGhISQkxMjNr2cM/Xuk8YVlVVYbfbcblcaLVakpKSiI2NVa8fZUrG1WrDXtuIo/YqstWBtaLWQy5UF633dZTWSJ3aTClOb4/q9raGV0LWxfoK02s6sY2SQnXQXllH3hXH5PEJHncKZ8+exWw2+9gueWtaKIrC+fPnOXPmDKmpqV0efbdarWzevJmvv/6aHTt2kJ6eft3n9iaFhYUkJyeTmJgIwOOPP05ubq5IyDfBHVMhA6xbt46dO3diNBo5cOBAl7d+gs5RFAWz2cyRI0fUVsepU6cYPHiwh+rdrQj8u5OQ2z1Fq9Wqo1sdrbKioqI8tKkVl4yj3oyjrgnH5SZwydiqLnts8rWNxXkmVCk8BGy+LtxyqAaNt8t0bCcu052c9gvpoApnXGBCa+xcP8Jtu+TuuXdU9hswYAAXL14kMjLyhnZOhw8f5sUXX2TRokW89NJLfeqK7s2ePXvIy8vjvffeA+CDDz6goKCAbdu2BSymIOLOa1k89NBDXLx40ef7WVlZzJ8/X328efNmrFYrmzZt6svw+i3uRNpR4L+uro6UlBQmTpzIxIkTmTBhQpfKZs3NzZSXl2MwGEhKSvJJLDdrlaU4XTivNOO40ozzSjOKw4n9TL2HfVRnx6dtoRJhdi+/vVAtOBUf0X4pIgzFyxUlpP34tDYmEuPciTf1e7TZbFRUVHDp0iXCw8NxuVwex6WNRqPazrFarbzxxhsUFBSQnZ1NWlraTa3VG4iE3CV3XkLuLmfOnGHWrFl+G0Rfs2YN+/btIzQ0lKSkJP7yl7/4zRvtdsXlclFeXq5qdRQXF+NyuXwE/q1WK/n5+URFRd3Q080bdz/aLap0Passj7habLiutuBsbEW22HFebsJ5tRW52YYEaAYbkOu8FeR8E7e2k7aIJjwUrG0JOnxSIuEZ3+v2z9La2kpZWRmRkZEkJyej1Wo92jnuvvu7775LZWUlNTU1zJgxgzfeeOO6h3L6mq+//pqNGzfy6aefAm2FD8Avf/nLQIYVLIiE3JFTp06RkpICwFtvvcXBgwfZs2ePX6792Wef8cADD6DT6Vi7di0AW7Zs8cu1+xOtra0UFxerWh2FhYW0trbyox/9iNmzZ2MymXpk8uptldXU1ITT6USv13tIk7r70R2nFxLjE4jVR+EyW3GZrcjNbUe+5RYbkk7TNrHRod/cabtimBFX+/eM/3UP2sgbH5VWFEV1cbmRyanFYuH111/n2LFjTJ8+nXPnzlFcXMyf/vQnjxnjQOF0OklNTWX//v2MHDkSk8nErl27AtbTDjJEQu7IwoULOXHiBBqNhtGjR7N9+3ZGjhzp93X27t3Lnj17+PDDD/1+7f7EH/7wB7744gs2bNhATU2NWknX1NQwevRoj9E7o9HYo6PgLS0tPlZZ4eHhNDc3ExERQVpaWrcOych2J4rFjmyxI9ucyBY7is2BYnMi251IGgnZbEETEUbk/Xfd8HotLS0cP36cgQMHkpiY2OXGaH5+PqtXr2bp0qWsXLmyz/wXb5ZPPvmEVatW4XK5WLZsGevWrQt0SMGCSMiBYO7cuSxatIilS5cGOpSgxmKxdDo+J8syp0+fVhP04cOHaW1t5fvf/76apO++++4eCfxXV1dTU1NDTEwMDoejU6ussLCwXpvEcY8XXrp0ibS0tC7bNK2trbz++uuUlJTw7rvvkpqa2isxBQMlJSX89Kc/pampCa1Wy7p161i0aFGgw/IXIiH7k+5sGGZlZXH48GH+/ve/i7E6P2K32/nPf/6j6nWUlpYSFhbmIfCflJR0w1aH2WymvLyc6OhoEhISPKrMnlpldRez2UxZWRkxMTEkJCR0GfNXX33FmjVreOqpp1ixYkXAquKPP/6YjRs3UlZWRmFhYa+J2LuP5aekpHD+/HkmTpxIWVlZf9mPEQm5L8nJyeGdd95h//79XVrl3Ap99Ya4XVAUhatXr3oI/FdUVDBixAh1NnrSpEnExsYiSRKtra2cO3eOxsZG7rrrLgwGQ7fW6I5VVnf73bIsU1lZSX19PWlpaV3G0NLSwmuvvUZpaSnZ2dnq3kegKCsrQ6PR8JOf/ITf/va3fnn9FRUV8cwzz1BYWIjL5WLy5Mn87W9/IyMjQ33OuHHj2LNnT8B/fj8hEnJfkZeXx0svvcTBgwd7Zba5N94Q/Q1FUThz5oyaoIuKimhoaCAmJobq6mrWrl3Lo48+2iMrK2+rrObmZiRJ8hhL6+z6TU1NlJeXM3jw4C6PnyuKwpdffsnatWtZtmwZL7zwQlD1iqdOnerX19+rr76K1WrFYrEQFxfnMY1RWFjIU089xbfffnvLm7xBhkjIfUVycjI2m009TTVlypReOZbt7zdEf0ZRFF544QWOHTvG7Nmzqaio4JtvvkGSJPUo+KRJkxgzZkyPkp73AY+OVlkGg4GGhgbMZjNpaWlERkZe9zotLS1s3LiR8vJysrOzSUpKuuWYegt/v/7sdjsmk4kBAwbw1VdfqX+HCxcuMHXqVHbs2MGUKVP8slYQII5O9xXfffddoEMQeCFJEs899xzjx4+/dlikXeD/8OHDFBYWsmXLFk6cOMGgQYM8Thl2ZX/kjU6nIzo62mMW2GazceHCBcrLy9HpdGg0Gk6fPt2pVZaiKBw6dIiXX36Z5557jrfeeisgFWF3D1X5k/r6epqbm3E4HFitVvR6PU1NTcyePZusrKz+lIy7jaiQg4TuvCFEhex/FEWhtrZW3TAsKiri4sWLJCYmegj8GwyGbiVpl8vF6dOn1ao4IiKiU6usgoICDh48iMPhoLGxkb/+9a9BP0Hh79ffvHnzePzxx6msrOTChQv8/ve/Z+bMmcydO5dVq1b5ZY0gQlTItxP/+te/+nxNIZXYVkkPGzaM+fPnqx98sixz8uRJ8vPz2bdvH5s2bcJut/sI/HtPXTQ0NHDixAlGjhxJSkqKmsAlSSIiIoKIiAiGDx+Ooig0Njayd+9eEhMTGTFiBIsXL+bpp59m+fLlff47CAQ7d+4kJCSExYsX43K5+MEPfsDu3bv597//TX19PTk5OUDbZnlmZmZgg+1DRIV8G+HPCkVIJd4cVqvVQ+D/22+/Ra/XM2HCBDIyMvjss8945JFHmDVrVpfypGazmfXr11NVVUV2djbx8fHq/wWLi4c3e/fuZcWKFdTV1TFw4EAyMzPV49GCbiM29foLvfGGELoDPUNRFK5cuUJ2djZbt25lzJgx1NXVMWrUKI9ThtHR0UiShKIofP7557zyyiv87Gc/49lnn+0v0wOC7iFaFv2FBQsWsGDBAr9e89y5c3zve9fEb+Li4igoKPDrGv0ZSZIYNGgQVquVoqIi4uLiVCfu/Px8Dhw4wJtvvonZbCY1NVVVcNu3bx+jRo0KWNxCCCu4EQlZILhFJEnykHDVaDQkJiaSmJjI4sWLgTZFuqNHj7Jv3z42bNgQ8Kp42rRpbN68WRXC2rx5sxDCCiJEQr5DGTlyJGfPnlUf19TU9IrY0p1OSEiIqgkdDEyfPl3995QpU/ymeCjwD6KJdYdiMpk4deoUlZWV2O12du/ezbx58/y6xrJlyxgyZIjHcVhB8PD+++8zc+bMQIch6IBIyHcoOp2Obdu28fDDD5OWlsZjjz3md93ap59+mry8PL9eU3BjHnroITIyMny+cnNz1edkZWWh0+lYsmRJACMVeCOmLAS9SlVVFXPmzPGbO4ug5/SmEJbguogpC4FA4EleXh6/+c1vOHjwoEjGQYhoWQgEdxDLly/HbDYzbdo0MjMzef755wMdkqADokLup8yYMYP8/Hx++MMf8o9//CPQ4fQKZ8+e5cknn6S2thZJkvjxj3/MypUrAx1WUCOEsIIbUSH3U9asWcMHH3wQ6DB6FZ1Ox+9+9zuOHz9Ofn4+b7/9NsePHw90WH3G+vXrGTt2LJmZmUyfPp3z588HOiRBDxEJ+TanqKiIsWPHYrVaaWlpIT09ndLSUh588MFuOWP0Jk888QT33nsvJ06cIC4ujj//+c9+vf7w4cOZMGECAAaDgbS0NM6dO+fXNYKZNWvWcPToUUpKSpgzZw6vvfZaoEMS9BDRsrjNMZlMzJs3j1dffRWLxcLSpUuDZu73o48+6rO1qqqq+Oabb7jnnnv6bM1A09EctaWlJSiFiQQ3h0jI/YANGzaozgtbt24NdDh9TnNzMwsXLuSPf/xjlw7O/ZF169axc+dOjEYjBw4cCHQ4gh4iWhb9ALfzgtlsxmq1BjqcPsXhcLBw4UKWLFnCo48+6vfrW61WJk+ezLhx40hPT+dXv/qV39foihsd8sjKyuLs2bMsWbKEbdu29WlsAv9zswdDBEGIJEn/B+wGEoDhiqIsb//+VGC1oihzAhheryG13aPvAK4oitIrFhPta+gVRWmWJCkE+AJYqShKfm+sd6tIkjQK+ERRlODoVwluCVEh3+ZIkvQk4FAUZRfwa8AkSdIDkiQdAj4GHpQkqUaSpIcDGmjvcB/w38ADkiSVtH/N8ucCShvN7Q9D2r+CooqRJCmlw8P5QHmgYhH4B1EhCwQ3QJIkLXAESAbeVhRlbYBDAkCSpP8FxgAyUA08ryjKnTNm0g8RCVkg6CaSJA0E9gIrFEUR4hwCvyNaFgJBN1EUpRE4AMwIdCyC/olIyAJBF0iSNLi9MkaSpHBgGqJXK+glxByyQNA1w4Ed7X1kDfA/iqL0T3EQQcARPWSBQCAIEkTLQiAQCIIEkZAFAoEgSBAJWSAQCIKE/wf0bqowcObfLAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "\n",
    "# 使用numpy.meshgrid生成x1,x2矩阵，矩阵的每一行为[-3, 3]，以0.1为间隔的数值\n",
    "x1 = np.arange(-3, 3, 0.1)\n",
    "x2 = np.arange(-3, 3, 0.1)\n",
    "x1, x2 = np.meshgrid(x1, x2)\n",
    "init_x = paddle.to_tensor([x1, x2])\n",
    "model = OptimizedFunction3D()\n",
    "\n",
    "# 绘制f_3d函数的三维图像\n",
    "fig = plt.figure()\n",
    "ax = plt.axes(projection='3d')\n",
    "ax.plot_surface(init_x[0], init_x[1], model(init_x), color='#f19ec2')\n",
    "ax.set_xlabel('x1')\n",
    "ax.set_ylabel('x2')\n",
    "ax.set_zlabel('f(x1,x2)')\n",
    "plt.savefig('opti-f-3d.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可视化不同优化器情况下参数变化轨迹。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/mpl_toolkits/mplot3d/proj3d.py:141: RuntimeWarning: invalid value encountered in true_divide\n",
      "  txs, tys, tzs = vecw[0]/w, vecw[1]/w, vecw[2]/w\n"
     ]
    }
   ],
   "source": [
    "from IPython.display import HTML\n",
    "\n",
    "labels = ['SGD', 'AdaGrad', 'RMSprop', 'Momentum', 'Adam']\n",
    "colors = ['#9c9d9f', '#f7d2e2', '#f19ec2', '#e86096', '#000000']\n",
    "\n",
    "anim = Visualization3D(*x_all_opts, z_values=z_all_opts, labels=labels, colors=colors, fig=fig, ax=ax)\n",
    "ax.legend(loc='upper left')\n",
    "HTML(anim.to_html5_video())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，对于我们构建的函数，有些优化器如Momentum在参数更新时成功逃离鞍点，其他优化器在本次实验中收敛到鞍点处没有成功逃离。但这并不证明Momentum优化器是最好的优化器，在模型训练时使用哪种优化器，还要结合具体的场景和数据具体分析。\n",
    "\n",
    "**动手练习7.6**  \n",
    "通过调用飞桨API, 实验比较不同优化算法在MNIST数据集上的收敛性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 7.4 参数初始化\n",
    "\n",
    "神经网络的参数学习是一个非凸优化问题。当使用梯度下降法来进行网络参数优化时，参数初始值的选取十分关键，关系到网络的优化效率和泛化能力。此外，由于神经网络优化时出现的对称权重现象（参见第4.4.1节），神经网络的参数不能初始化为相同的值，需要有一定的差异性。\n",
    "\n",
    "常用的参数初始化的方式通常有以下三种：\n",
    "* 随机初始化：最常用的参数初始化策略，通过一个随机采样函数来生成每个参数的初始值。\n",
    "* 预训练初始化：一种在实践中经常使用的初始化策略，如果目标任务的训练数据不足，可以使用一个已经在大规模数据上训练过的模型作为参数初始值。预训练模型在目标任务上的学习过程也称为精调Fine-Tuning。\n",
    "* 固定值初始化：对于神经网络中的某些重要参数，可以根据先验知识来初始化。比如对于使用ReLU激活函数的全连接层，其偏置通常可以设为比较小的正数（比如0.01），从而确保这一层的神经元的梯度不为0，避免死亡ReLU现象。\n",
    "\n",
    "虽然预训练初始化通常具有更好的收敛性和泛化性，但是灵活性不够，不能在目标任务上任意地调整网络结构。因此，好的随机初始化方法对训练神经网络模型来说依然十分重要。在本节我们主要介绍两种随机初始化方法：基于固定方差的参数初始化和基于方差缩放的参数初始化。\n",
    "\n",
    "### 7.4.1 基于固定方差的参数初始化\n",
    "\n",
    "一种最简单的随机初始化方法是从一个固定均值(通常为 0)和方差$\\sigma^2$的分布中采样来生成参数的初始值。基于固定方差的参数初始化方法主要有高斯分布初始化和均匀分布初始化两种：\n",
    "\n",
    "* 高斯分布初始化：使用一个高斯分布$\\mathscr{N}(0, \\sigma^2)$对每个参数进行随机初始化。\n",
    "* 均匀分布初始化：在一个给定的区间$[-r, r]$内采用均匀分布来初始化。\n",
    "\n",
    "高斯分布和均匀分布初始化的实现方式可以参考第4.4.1节参数初始化代码。\n",
    "\n",
    "### 7.4.2 基于方差缩放的参数初始化\n",
    "\n",
    "初始化一个深度网络时，为了缓解梯度消失或爆炸问题，我们尽可能保持每个神经元的输入和输出的方差一致，根据神经元的连接数量来自适应地调整初始化分布的方差，这类方法称为方差缩放（Variance Scaling）。\n",
    "\n",
    "\n",
    "Xavier初始化是参数初始化中常用的方法，根据每层的神经元数量来自动计算初始化参数方差。\n",
    "在计算出参数的理想方差后，可以通过高斯分布或均匀分布来随机初始化参数。若神经元采用Tanh函数，并采用高斯分布来随机初始化参数，连接权重$w_i^{(l)}$可以按$\\mathscr{N}(0, \\frac{2}{M_{l-1} + M_l})$的高斯分布进行初始化，其中$M_{l-1}$是第$l-1$层神经元个数。\n",
    "\n",
    "**笔记**\n",
    "\n",
    "Xavier初始化公式推导可参考《神经网络与深度学习》7.3.2.1 Xavier初始化。\n",
    "\n",
    "本节动手实现Xavier初始化，并观察其效果。\n",
    "\n",
    "#### 7.4.2.1 模型构建\n",
    "\n",
    "首先定义xavier_normal_std函数，根据$l$层和$l-1$层神经元的数量计算理想标准差。值得注意的是，在paddle.normal API中，通过指定标准差的值来生成符合正态分布的张量，因此，这里需要计算标准差。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def xavier_normal_std(input_size, output_size):\n",
    "    return np.sqrt(2 / (input_size + output_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**笔记**\n",
    "\n",
    "Xavier初始化适用于Logistic激活函数和Tanh激活函数，对于不同激活函数，高斯分布的方差和均匀分布的$r$值计算是不同的。xavier_normal_std定义针对Tanh激活函数的情况。\n",
    "\n",
    "定义一个全连接前馈网络（即多层感知器）MLP算子，实例化网络时可以通过layers_size指定网络每层神经元的数量，通过init_fn_name指定网络中参数初始化方法(Xavier高斯分布初始化、Xavier均匀分布初始化或$\\mathscr{N}(0, 1)$高斯分布初始化)，init_fn指定计算初始化时均值或数值范围的函数，act_fn指定激活函数。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class MLP(nn.Layer):\n",
    "    def __init__(self, layers_size, init_fn_name, init_fn, act_fn):\n",
    "        \"\"\"\n",
    "        多层网络初始化\n",
    "        输入：\n",
    "            - layers_size: 每层神经元的数量\n",
    "            - init_fn_name: 网络中参数初始化方法，可以为 'normal'或'uniform'\n",
    "            - init_fn: 函数，用来计算高斯分布标准差或均匀分布r值\n",
    "            - act_fn: 激活函数\n",
    "        \"\"\"\n",
    "        super(MLP, self).__init__()\n",
    "        self.linear = nn.Sequential()\n",
    "        self.num_layers = len(layers_size) - 1\n",
    "        for i in range(self.num_layers):\n",
    "            input_size, output_size = layers_size[i], layers_size[i + 1]\n",
    "            if init_fn_name == 'normal':\n",
    "                # Xavier高斯分布初始化，计算方差\n",
    "                self.linear.add_sublayer(str(i), nn.Linear(input_size, output_size,\n",
    "                                           weight_attr=nn.initializer.Normal(mean=0, std=init_fn(input_size, output_size))))\n",
    "            elif init_fn_name == 'uniform':\n",
    "                r = init_fn(input_size, output_size)\n",
    "                self.linear.add_sublayer(str(i), nn.Linear(input_size, output_size, weight_attr=nn.initializer.Uniform(low=-r, high=r)))\n",
    "            else:\n",
    "                self.linear.add_sublayer(str(i), nn.Linear(input_size, output_size, weight_attr=nn.initializer.Normal()))\n",
    "        self.act_fn = act_fn()\n",
    "        self.z = {}\n",
    "\n",
    "    def __call__(self, X):\n",
    "        return self.forward(X)\n",
    "\n",
    "    def forward(self, X):\n",
    "        \"\"\"\n",
    "        前向计算\n",
    "        \"\"\"\n",
    "        y = X\n",
    "        for num_layer in range(self.num_layers):\n",
    "            y = self.linear[num_layer](y)\n",
    "            if num_layer != self.num_layers - 1:\n",
    "                y = self.act_fn(y)\n",
    "            self.z[num_layer] = y\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.4.2.2 观察模型神经元的方差变化\n",
    "**高斯分布初始化**  定义网络每层神经元的数量，指定激活函数和参数初始化方式，通过Xavier高斯分布初始化网络。代码实现如下:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "# 定义网络每层神经元的数量\n",
    "layers_size = [100, 200, 400, 300, 200, 100]\n",
    "# 指定激活函数\n",
    "activate_fn = paddle.nn.Tanh\n",
    "# 指定参数初始化方式\n",
    "init_fn_name = 'normal'\n",
    "\n",
    "model = MLP(layers_size, init_fn_name, init_fn=xavier_normal_std, act_fn=activate_fn)\n",
    "inputs = paddle.normal(shape=[1, 100], std=0.1)\n",
    "y = model(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "打印每层神经元输出的方差，观察每层的方差值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for i in range(len(model.z) - 1):\n",
    "    print('layer %d: , %f'%(i, model.z[i].numpy().var()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，Xavier初始化可以尽量保持每个神经元的输入和输出方差一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**均匀分布初始化** 若采用区间为$[-r, r]$的均匀分布来初始化$w_i^{(l)}$，则$r$的取值为$\\sqrt{\\frac{6}{M_{l-1} + M_l}}$。定义xavier_uniform_r，计算均匀分布$r$的值。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def xavier_uniform_r(input_size, output_size):\n",
    "    return np.sqrt(6 / (input_size + output_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义网络每层神经元的数量，通过Xavier均匀分布初始化网络。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "# 指定激活函数\n",
    "activate_fn = paddle.nn.Tanh\n",
    "# 指定参数初始化方式\n",
    "init_fn_name = 'uniform'\n",
    "\n",
    "model = MLP(layers_size, init_fn_name, init_fn=xavier_uniform_r, act_fn=activate_fn)\n",
    "inputs = paddle.normal(shape=[1, 100], std=0.1)\n",
    "y = model(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "打印每层神经元输出的方差，观察每层的方差值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for i in range(len(model.z) - 1):\n",
    "    print('layer %d: , %f'%(i, model.z[i].numpy().var()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.4.2.3 观察模型训练收敛性\n",
    "\n",
    "为了进一步验证Xavier初始化的效果，我们在一个简单的二分类任务上来训练MLP模型，并观察模型收敛情况。\n",
    "\n",
    "**构建数据集** 这里使用在第3.1.1中定义的make_moons函数构建一个简单的二分类数据集。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nndl import make_moons\n",
    "\n",
    "class MoonsDataset(io.Dataset):\n",
    "    def __init__(self, mode='train', num_samples=300, num_train=200):\n",
    "        super(MoonsDataset, self).__init__()\n",
    "        X, y = make_moons(n_samples=num_samples, shuffle=True, noise=0.5)\n",
    "        if mode == 'train':\n",
    "            self.X, self.y = X[:num_train], y[:num_train]\n",
    "        else:\n",
    "            self.X, self.y = X[num_train:], y[num_train:]\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        return self.X[idx], self.y[idx]\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "创建训练和验证集，构建DataLoader。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "train_dataset = MoonsDataset(mode='train')\n",
    "dev_dataset = MoonsDataset(mode='dev')\n",
    "train_loader = io.DataLoader(train_dataset, batch_size=10, shuffle=True)\n",
    "dev_loader = io.DataLoader(dev_dataset, batch_size=10, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义五层MLP，分别以Xavier初始化和标准高斯分布初始化方式对网络进行初始化，训练100回合，对比两个模型的训练损失变化情况。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import nndl\n",
    "\n",
    "paddle.seed(0)\n",
    "np.random.seed(0)\n",
    "\n",
    "# 定义网络每层神经元的数量\n",
    "layers_size = [2, 300, 500, 700, 400, 1]\n",
    "# 指定激活函数\n",
    "activate_fn = paddle.nn.Tanh\n",
    "\n",
    "# 指定参数初始化方式为Xavier高斯分布初始化\n",
    "init_fn_name = 'normal'\n",
    "model1 = MLP(layers_size, init_fn_name, init_fn=xavier_normal_std, act_fn=activate_fn)\n",
    "opt1 = optimizer.SGD(learning_rate=0.005, parameters=model1.parameters())\n",
    "loss_fn = F.binary_cross_entropy_with_logits\n",
    "m = nndl.Accuracy(is_logist=True)\n",
    "runner1 = RunnerV3(model1, opt1, loss_fn, m)\n",
    "runner1.train(train_loader, dev_loader, num_epochs=100, eval_steps=400, log_steps=0)\n",
    "\n",
    "# 指定参数初始化方式为N(0, 1)高斯分布初始化\n",
    "init_fn_name = 'basic'\n",
    "model2 = MLP(layers_size, init_fn_name, None, act_fn=activate_fn)\n",
    "opt2 = optimizer.SGD(learning_rate=0.005, parameters=model2.parameters())\n",
    "runner2 = RunnerV3(model2, opt2, loss_fn, m)\n",
    "runner2.train(train_loader, dev_loader, num_epochs=100, eval_steps=400, log_steps=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots()\n",
    "plt.plot(runner1.train_epoch_losses, label='xavier initializer', c='#e4007f', linestyle='--')\n",
    "plt.plot(runner2.train_epoch_losses, label='N(0,1) initializer', c='#f19ec2')\n",
    "ax.set_xlabel('epoch')\n",
    "ax.set_ylabel('loss')\n",
    "plt.legend(fontsize='large')\n",
    "plt.savefig('opti-xavier.pdf')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，使用Xavier初始化，模型的损失相对较小，模型效果更好。\n",
    "\n",
    "**动手练习7.7**\n",
    "\n",
    "尝试实现He初始化，并将上面MLP算子的激活函数改为ReLU，观察其效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 7.5 逐层规范化\n",
    "\n",
    "逐层规范化（Layer-wise Normalization）是将传统机器学习中的数据规范化方法应用到深度神经网络中，对神经网络中隐藏层的输入进行规范化，从而使得网络更容易训练。\n",
    "\n",
    "在深度神经网络中，一个神经层的输入是之前神经层的输出。给定一个神经层$l$，它之前的神经层$(1, \\cdots, l-1$)的参数变化会导致其输入的分布发生较大的改变。从机器学习角度来看，如果一个神经层的输入分布发生了改变，那么其参数需要重新学习，这种现象叫作内部协变量偏移（Internal Covariate Shift）。 为了缓解这个问题，我们可以对每一个神经层的输入进行规范化操作，使其分布保持稳定。\n",
    "\n",
    "下面介绍两种比较常用的逐层规范化方法：批量规范化（Batch Normalization）和层规范化（Layer Normalization）。\n",
    "\n",
    "### 7.5.1 批量规范化\n",
    "\n",
    "对于一个深度神经网络，为了提高优化效率，要使得第$l$层的净输入$\\bm z^{(l)}$的分布一致，比如都规范化到标准正态分布。在实践中规范化操作一般应用在线性层和激活函数之间。而为了提高规范化效率，一般使用标准化将净输入$\\bm z^{(l)}$的每一维都规范化到标准正态分布。\n",
    "\n",
    "$$\n",
    "\\hat{\\bm z}^{(l)} = \\frac{\\bm z^{(l)} - \\bm \\mu_{\\mathcal B}}{\\sqrt{\\bm \\sigma_{\\mathcal B}^2 + \\epsilon}},\n",
    "$$\n",
    "其中$\\bm \\mu_{\\mathcal B}$、$\\bm \\sigma_{\\mathcal B}^2$为小批量样本的均值和方差。\n",
    "\n",
    "对净输入$\\bm z^{(l)}$的标准规范化会使得其取值集中到0附近，如果使用Sigmoid型激活函数时，这个取值区间刚好是接近线性变换的区间，减弱了神经网络的非线性性质。因此，为了使得规范化不对网络的表示能力造成负面影响，可以通过一个附加的缩放和平移变换改变取值区间。则有：\n",
    "$$\n",
    "\\hat{\\bm z}^{(l)} \\triangleq BN_{\\bm \\gamma, v \\beta}(\\bm z^{(l)}) = \\frac{\\bm z^{(l)} - \\bm \\mu_{\\mathcal B}}{\\sqrt{\\bm \\sigma_{\\mathcal B}^2 + \\epsilon}} \\odot \\bm \\gamma + \\bm \\beta.\n",
    "$$\n",
    "\n",
    "#### 7.5.1.1 BatchNorm算子\n",
    "\n",
    "下面定义BatchNorm算子，实现批量规范化。在实现批量规范化时，在训练过程中的均值和方差可以动态计算，但在测试时需要保存固定，否则模型输出就会受到同一批次中其他样本的影响。因此，在训练时需要将每一批次样本的均值和方差以移动平均值的方式记录下来，预测时使用整个训练集上的均值和方差（也就是保存的移动平均值）进行规范化。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class BatchNorm(nn.Layer):\n",
    "    def __init__(self, num_features, eps=1e-7, momentum=0.9, gamma=1.0, beta=0.0):\n",
    "        \"\"\"\n",
    "        批量规范化初始化\n",
    "        输入:\n",
    "            - num_features: 输入特征数\n",
    "            - eps: 保持数值稳定性而设置的常数\n",
    "            - momentum: 用于计算移动平均值\n",
    "            - gamma: 缩放的参数\n",
    "            - beta: 平移的参数\n",
    "        \"\"\"\n",
    "        super(BatchNorm, self).__init__()\n",
    "        shape = (1, num_features)\n",
    "        self.gamma = paddle.to_tensor(gamma, dtype='float32')\n",
    "        self.beta = paddle.to_tensor(beta, dtype='float32')\n",
    "        self.moving_mean = paddle.zeros(shape)\n",
    "        self.moving_variance = paddle.ones(shape)\n",
    "        self.eps = eps\n",
    "        self.momentum = momentum\n",
    "\n",
    "    def __call__(self, X, train_mode=True):\n",
    "        return self.forward(X, train_mode)\n",
    "\n",
    "    def forward(self, X, train_mode=True):\n",
    "        if not train_mode:\n",
    "            X = (X - self.moving_mean) / paddle.sqrt(self.moving_variance + self.eps)\n",
    "        else:\n",
    "            assert len(X.shape) in (2, 4)\n",
    "            if len(X.shape) == 2:\n",
    "                # 对于Linear层\n",
    "                mean = paddle.mean(X, axis=0)\n",
    "                var = ((X - mean) ** 2).mean(axis=0)\n",
    "            else:\n",
    "                # 对于卷积层\n",
    "                mean = paddle.mean(X, axis=[0, 2, 3], keepdim=True)\n",
    "                var = ((X - mean) ** 2).mean(axis=[0, 2, 3], keepdim=True)\n",
    "            X = (X - mean) / paddle.sqrt(var, self.eps)\n",
    "            # 保存移动平均值\n",
    "            self.moving_mean = self.momentum * self.moving_mean + (1. - self.momentum) * mean\n",
    "            self.moving_variance = self.momentum * self.moving_variance + (1. - self.momentum) * var\n",
    "        y = self.gamma * X + self.beta\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.5.1.2 支持逐层规范化的MLP算子\n",
    "\n",
    "重新定义MLP算子，加入逐层规范化功能。初始化网络时新增三个参数：norm_name指定使用哪一种逐层规范化（默认为None）、gamma和beta为缩放和平移变换的参数。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class MLP(nn.Layer):\n",
    "    def __init__(self, layers_size, init_fn_name, init_fn, act_fn, norm_name=None, gamma=None, beta=None):\n",
    "        \"\"\"\n",
    "        多层网络初始化\n",
    "        输入：\n",
    "            - layers_size: 每层神经元的数量\n",
    "            - init_fn_name: 网络中参数初始化方法\n",
    "            - init_fn: 计算高斯分布标准差或均匀分布r值\n",
    "            - act_fn: 激活函数\n",
    "            - norm_name: 使用哪一种逐层规范化\n",
    "            - gamma、beta: 缩放和平移变换的参数\n",
    "        \"\"\"\n",
    "        super(MLP, self).__init__()\n",
    "        self.linear = paddle.nn.Sequential()\n",
    "        self.normalization = {}\n",
    "        self.num_layers = len(layers_size) - 1\n",
    "        for i in range(self.num_layers):\n",
    "            input_size, output_size = layers_size[i], layers_size[i + 1]\n",
    "            if init_fn_name == 'normal':\n",
    "                # Xavier高斯分布初始化，计算方差\n",
    "                self.linear.add_sublayer(str(i), nn.Linear(input_size, output_size,\n",
    "                                           weight_attr=nn.initializer.Normal(mean=0, std=init_fn(input_size, output_size))))\n",
    "            elif init_fn_name == 'uniform':\n",
    "                r = init_fn(input_size, output_size)\n",
    "                self.linear.add_sublayer(str(i), nn.Linear(input_size, output_size, weight_attr=nn.initializer.Uniform(low=-r, high=r)))\n",
    "            else:\n",
    "                self.linear.add_sublayer(str(i), nn.Linear(input_size, output_size, weight_attr=nn.initializer.Normal()))\n",
    "            # 判断是否使用逐层规范化，以及使用哪一种逐层规范化\n",
    "            if norm_name == 'bn':\n",
    "                self.normalization[i] = BatchNorm(output_size, gamma=gamma[i], beta=beta[i])\n",
    "            elif norm_name == 'ln':\n",
    "             # LayerNorm：对一个中间层的所有神经元进行规范化\n",
    "                self.normalization[i] = LayerNorm(gamma=gamma[i], beta=beta[i])\n",
    "        self.act_fn = act_fn()\n",
    "        self.norm_name = norm_name\n",
    "        self.z = {}\n",
    "\n",
    "    def __call__(self, X, train_mode=True):\n",
    "        return self.forward(X, train_mode)\n",
    "\n",
    "    def forward(self, X, train_mode=True):\n",
    "        y = X\n",
    "        for num_layer in range(self.num_layers):\n",
    "            y = self.linear[num_layer](y)\n",
    "            if num_layer != self.num_layers - 1:\n",
    "                if self.norm_name == 'bn':\n",
    "                    y = self.normalization[num_layer](y, train_mode)\n",
    "                elif self.norm_name == 'ln':\n",
    "                    y = self.normalization[num_layer](y)\n",
    "                # 为了展示逐层规范化后的输出的均值和方差，使用z[num_layer]进行记录\n",
    "                self.z[num_layer] = y\n",
    "                y = self.act_fn(y)\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "因为批量规范化是对一个中间层的单个神经元进行规范化操作，所以要求小批量样本的数量不能太小，否则难以计算单个神经元的统计信息。所以我们使用paddle.randn随机生成一组形状为(200, 100)的数据, 打印数据送入网络前的均值与标准差。再分别定义使用批量规范化和不使用批量规范化的五层线性网络，分别打印网络第四层的均值与标准差，对比结果。\n",
    "\n",
    "#### 7.5.1.3 内部协变量偏移实验\n",
    "\n",
    "下面我们构建两个模型：model1不使用批量规范化，model2使用批量规范化，观察批量规范化是否可以缓解内部协变量偏移问题。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "# 定义网络每层神经元的数量\n",
    "layers_size = [100, 200, 400, 300, 2, 2]\n",
    "\n",
    "data = paddle.randn(shape=[200, 100])\n",
    "print('data mean: ', data.numpy().mean())\n",
    "print('data std: ', data.numpy().std())\n",
    "\n",
    "activate_fn = paddle.nn.Tanh\n",
    "model1 = MLP(layers_size, 'basic', None, act_fn=activate_fn)\n",
    "output = model1(data)\n",
    "print('no batch normalization: ')\n",
    "print('model output mean: ', model1.z[3].numpy().mean(axis=0))\n",
    "print('model output std:', model1.z[3].numpy().std(axis=0))\n",
    "\n",
    "gamma = [1, 1, 1, 1, 1]\n",
    "beta = [0, 0, 0, 0, 0]\n",
    "model2 = MLP(layers_size, 'basic', None, act_fn=activate_fn, norm_name='bn', gamma=gamma, beta=beta)\n",
    "output = model2(data)\n",
    "print('with batch normalization: ')\n",
    "print('model output mean: ', model2.z[3].numpy().mean(axis=0))\n",
    "print('model output std:', model2.z[3].numpy().std(axis=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，在经过多层网络后，网络输出的均值和标准差已经发生偏移。而当我们指定批量规范化的均值和标准差为0,1时，网络输出的均值和标准差就会变为0,1。\n",
    "\n",
    "当我们指定$\\bm \\gamma$和$\\bm \\beta$时，网络输出的标准差和均值就变为$\\bm \\gamma$和$\\bm \\beta$的值。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "gamma = [1, 2, 3, 5, 4]\n",
    "beta = [3, 2, 1, 2, 2]\n",
    "model3 = MLP(layers_size, 'basic', None, act_fn=activate_fn, norm_name='bn', gamma=gamma, beta=beta)\n",
    "output = model3(data)\n",
    "print('batch normalization with different gamma and beta for different layer: ')\n",
    "print('output means with bn 0: ', model3.z[0].numpy().mean())\n",
    "print('output stds with bn 0: ', model3.z[0].numpy().std())\n",
    "print('output means with bn 3: ', model3.z[3].numpy().mean())\n",
    "print('output stds with bn 3: ', model3.z[3].numpy().std())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.5.1.4 均值和方差的移动平均计算实验\n",
    "\n",
    "下面测试批量规范化中训练样本均值和方差的移动平均值计算。使网络前向迭代50个回合，这个前向计算并不涉及网络训练与梯度更新，只是模拟网络训练时批量规范化中训练样本的均值和方差用移动平均计算的过程。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "epochs = 50\n",
    "for epoch in range(epochs):\n",
    "    inputs = paddle.randn(shape=[200, 100])\n",
    "    output = model3(data)\n",
    "\n",
    "# 打印批量规范化中训练样本均值和方差的移动平均值\n",
    "print('batch norm 3 moving mean: ', model3.normalization[3].moving_mean.numpy())\n",
    "print('batch norm 3 moving variance: ', model3.normalization[3].moving_variance.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "开启测试模式，使用训练集的移动平均值作为测试集批量规范化的均值和标准差。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "inputs_test = paddle.randn(shape=[5, 100])\n",
    "output = model3(inputs_test, train_mode=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.5.1.5 在MNIST数据集上使用带批量规范化的卷积网络\n",
    "\n",
    "批量规范化的提出是为了解决内部协方差偏移问题，但后来发现其主要优点是更平滑的优化地形，以及使梯度变得更加稳定，从而提高收敛速度。\n",
    "\n",
    "为验证批量规范化的有效性，本节使用飞桨API快速搭建一个多层卷积神经网络。在MNIST数据集上，观察使用批量规范化的网络是否相对于没有使用批量规范化的网络收敛速度更快。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from paddle.nn import Conv2D, MaxPool2D, Linear, BatchNorm2D\n",
    "\n",
    "# 多层卷积神经网络实现\n",
    "class MultiConvLayerNet(nn.Layer):\n",
    "    def __init__(self, use_bn=False):\n",
    "        super(MultiConvLayerNet, self).__init__()\n",
    "\n",
    "        # 定义卷积层，输出特征通道out_channels设置为20，卷积核的大小kernel_size为5，卷积步长stride=1，padding=2\n",
    "        self.conv1 = Conv2D(in_channels=1, out_channels=20, kernel_size=5, stride=1, padding=2)\n",
    "        # 定义汇聚层，窗口的大小为2，步长为2\n",
    "        self.max_pool1 = MaxPool2D(kernel_size=2, stride=2)\n",
    "        # 定义卷积层，输出特征通道out_channels设置为20，卷积核的大小kernel_size为5，卷积步长stride=1，padding=2\n",
    "        self.conv2 = Conv2D(in_channels=20, out_channels=20, kernel_size=5, stride=1, padding=2)\n",
    "        # 定义汇聚层，窗口的大小为2，步长为2\n",
    "        self.max_pool2 = MaxPool2D(kernel_size=2, stride=2)\n",
    "        # 定义一层全连接层，输出维度是10\n",
    "        self.fc = Linear(980, 10)\n",
    "        if use_bn:\n",
    "            # 定义批量规范化层\n",
    "            self.batch_norm1 = BatchNorm2D(num_features=20)\n",
    "            self.batch_norm2 = BatchNorm2D(num_features=20)\n",
    "        self.use_bn = use_bn\n",
    "\n",
    "    # 定义网络前向计算过程\n",
    "    def forward(self, inputs):\n",
    "        x = self.conv1(inputs)\n",
    "        if self.use_bn:\n",
    "            x = self.batch_norm1(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.max_pool1(x)\n",
    "        x = self.conv2(x)\n",
    "        if self.use_bn:\n",
    "            x = self.batch_norm2(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.max_pool2(x)\n",
    "        x = paddle.reshape(x, [x.shape[0], 980])\n",
    "        x = self.fc(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "实例化网络并进行训练。model1不使用批量规范化，model2使用批量规范化。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nndl import Accuracy\n",
    "\n",
    "paddle.seed(0)\n",
    "# 确保从paddle.vision.datasets.MNIST中加载的图像数据是np.ndarray类型\n",
    "paddle.vision.image.set_image_backend('cv2')\n",
    "\n",
    "# 使用MNIST数据集\n",
    "train_dataset = MNIST(mode='train', transform=transform)\n",
    "train_loader = io.DataLoader(train_dataset, batch_size=64, shuffle=True)\n",
    "dev_dataset = MNIST(mode='test', transform=transform)\n",
    "dev_loader = io.DataLoader(train_dataset, batch_size=64)\n",
    "model1 = MultiConvLayerNet(use_bn=False)\n",
    "opt1 = paddle.optimizer.Adam(learning_rate=0.01, parameters=model1.parameters())\n",
    "loss_fn = F.cross_entropy\n",
    "metric = Accuracy()\n",
    "runner1 = RunnerV3(model1, opt1, loss_fn, metric)\n",
    "print('train network without batch normalization')\n",
    "runner1.train(train_loader, dev_loader, num_epochs=5, log_steps=0, eval_steps=300)\n",
    "\n",
    "model2 = MultiConvLayerNet(use_bn=True)\n",
    "opt2 = paddle.optimizer.Adam(learning_rate=0.01, parameters=model2.parameters())\n",
    "runner2 = RunnerV3(model2, opt2, loss_fn, metric)\n",
    "print('train network with batch normalization')\n",
    "runner2.train(train_loader, dev_loader, num_epochs=5, log_steps=0, eval_steps=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "对比model1和model2在验证集上损失和准确率的变化情况。从输出结果看，使用批量规范化的网络收敛速度会更好。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10, 4))\n",
    "ax1 = plt.subplot(121)\n",
    "ax1.plot(np.array(runner1.dev_losses)[:, 1], label='no bn', c='#e4007f', linestyle='--')\n",
    "ax1.plot(np.array(runner2.dev_losses)[:, 1], label='with bn', c='#f19ec2')\n",
    "ax1.set_xlabel('step')\n",
    "ax1.set_ylabel('loss')\n",
    "plt.legend(fontsize='x-large')\n",
    "ax2 = plt.subplot(122)\n",
    "ax2.plot(runner1.dev_scores, label='no bn', c='#e4007f', linestyle='--')\n",
    "ax2.plot(runner2.dev_scores, label='with bn', c='#f19ec2')\n",
    "ax2.set_xlabel('step')\n",
    "ax2.set_ylabel('accuracy')\n",
    "plt.legend(fontsize='x-large')\n",
    "plt.savefig('opti-acc.pdf')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 7.5.2 层规范化\n",
    "\n",
    "层规范化（Layer Normalization）和批量规范化是非常类似的方法，它们的区别在于批量规范化对中间层的单个神经元进行规范化操作，而层规范化对一个中间层的所有神经元进行规范化。\n",
    "\n",
    "层规范化定义为\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\hat{\\bm z}^{(l)} &= \\frac{\\bm z^{(l)} - \\mu^{(l)}}{\\sqrt{\\sigma^{(l)^2} + \\epsilon}} \\odot \\bm \\gamma + \\bm \\beta,  \\\\\n",
    "&\\triangleq LN_{\\bm \\gamma, \\bm \\beta}(\\bm z^{(l)}),\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "其中$\\bm z^{(l)}$为第$l$层神经元的净输入， $\\bm \\gamma$和$\\bm \\beta$分别代表缩放和平移的参数向量，和$\\bm z^{(l)}$维数相同。$\\mu^{(l)}$和$\\sigma^{(l)^2}$分别为$\\bm z^{(l)}$的均值和方差。\n",
    "\n",
    "根据上面的公式可以看出，对于$K$个样本的一个小批量合集$\\bm z^{(l)} = [\\bm z^{(1, l)}; ...; \\bm z^{(K, l)}]$，层规范化是对矩阵$\\bm z^{(l)}$的每一列进行规范化，而批量规范化是对每一行进行规范化。一般而言，批量规范化是一种更好的选择。当小批量样本数量比较小时，可以选择层规范化。\n",
    "\n",
    "#### 7.5.2.1 LayerNorm算子\n",
    "\n",
    "定义LayerNorm实现层规范化算子。与批量规范化不同，层规范化对每个样本的所有特征进行规范化。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class LayerNorm(nn.Layer):\n",
    "    def __init__(self, eps=1e-7,  gamma=1.0, beta=0.0):\n",
    "        \"\"\"\n",
    "        层规范化初始化\n",
    "        输入:\n",
    "            - eps: 保持数值稳定性而设置的常数\n",
    "            - gamma: 缩放的参数\n",
    "            - beta: 平移的参数\n",
    "        \"\"\"\n",
    "        super().__init__(self.__class__.__name__)\n",
    "        self.gamma = paddle.to_tensor(gamma, dtype='float32')\n",
    "        self.beta = paddle.to_tensor(beta, dtype='float32')\n",
    "        self.eps = eps\n",
    "\n",
    "    def forward(self, X):\n",
    "        # 层规范化对每个样本的每个特征进行规范化\n",
    "        assert len(X.shape) in (2, 3, 4)\n",
    "        if len(X.shape) == 4:\n",
    "            mean = paddle.mean(X, axis=[1, 2, 3], keepdim=True)\n",
    "            var = ((X - mean) ** 2).mean(axis=[1, 2, 3], keepdim=True)\n",
    "        else:\n",
    "            mean = paddle.mean(X, axis=-1, keepdim=True)\n",
    "            var = ((X - mean) ** 2).mean(axis=-1, keepdim=True)\n",
    "        X = (X - mean) / paddle.sqrt(var, self.eps)\n",
    "        y = self.gamma * X + self.beta\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.5.2.2 层规范化的验证实验\n",
    "随机初始化一组形状为（10，100）的数据，输入带有层规范化的前馈神经网络中，得到网络输出并打印输出的标准差和均值。指定$\\bm \\gamma$和$\\bm \\beta$，从输出结果看，网络输出的标准差和均值变为$\\bm \\gamma$和$\\bm \\beta$的值。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "# 定义网络每层神经元的数量\n",
    "layers_size = [100, 200, 400, 300, 2, 2]\n",
    "\n",
    "# 随机生成数据\n",
    "data = paddle.randn(shape=[10, 100])\n",
    "activate_fn = paddle.nn.Tanh\n",
    "gamma = [1, 2, 3, 5, 4]\n",
    "beta = [3, 2, 1, 2, 2]\n",
    "model = MLP(layers_size, 'basic', None, act_fn=activate_fn, norm_name='ln', gamma=gamma, beta=beta)\n",
    "output = model(data)\n",
    "print('layer normalization with different gamma and beta for different layer: ')\n",
    "print('output means with ln 0: ', model.z[0].numpy().mean(axis=-1))\n",
    "print('output stds with ln 0: ', model.z[0].numpy().std(axis=-1))\n",
    "print('output means with ln 1: ', model.z[3].numpy().mean(axis=-1))\n",
    "print('output stds with ln 1: ', model.z[3].numpy().std(axis=-1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "因为层规范化是对每个样本的每个通道做规范化，不需要存储训练数据的均值和方差的移动平均值，所以这里不需要多轮迭代累计移动平均值再做测试。而随机生成测试数据经过带层规范化的神经网络和上述代码实现方式相同，这里不再重复展示。\n",
    "\n",
    "**动手练习7.8** \n",
    "\n",
    "尝试在MNIST数据集上对比使用层规范化的网络与没有使用层规范化的网络在收敛速度上的区别。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 7.6 网络正则化方法\n",
    "\n",
    "由于深度神经网络的复杂度比较高，并且拟合能力很强，很容易在训练集上产生过拟合，因此在训练深度神经网络时，也需要通过一定的正则化方法来改进网络的泛化能力。\n",
    "\n",
    "正则化（Regularization）是一类通过限制模型复杂度，从而避免过拟合、提高泛化能力的方法，比如引入约束、增加先验、提前停止等。\n",
    "\n",
    "为了展示不同正则化方法的实现方式和效果，本节构建一个小数据集和多层感知器来模拟一个过拟合的实验场景，并实现$\\ell_2$正则化、权重衰减和暂退法，观察这些正则化方法是否可以缓解过拟合现象。\n",
    "\n",
    "\n",
    "### 7.6.1 数据集构建\n",
    "\n",
    "首先使用数据集构建函数make_moons来构建一个小的数据集，生成300个样本，其中200个作为训练数据，100个作为测试数据。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "\n",
    "# 采样300个样本\n",
    "n_samples = 300\n",
    "num_train = 200\n",
    "\n",
    "# 根据make_moons生成二分类数据集\n",
    "data_X, data_y = make_moons(n_samples=n_samples, shuffle=True, noise=0.5)\n",
    "X_train, y_train = data_X[:num_train], data_y[:num_train]\n",
    "X_test, y_test = data_X[num_train:], data_y[num_train:]\n",
    "\n",
    "y_train = y_train.reshape([-1, 1])\n",
    "y_test = y_test.reshape([-1, 1])\n",
    "print('train dataset X shape: ', X_train.shape)\n",
    "print('train dataset y shape: ', y_train.shape)\n",
    "print(X_train[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 7.6.2 模型构建\n",
    "\n",
    "为了更好地展示正则化方法的实现机理，本节使用本书自定义的Op类来构建一个全连接前馈网络（即多层感知器）MLP_3L。MLP_3L是一个三层感知器，使用ReLU激活函数，最后一层输出层为线性层，即输出对率。\n",
    "\n",
    "首先，我们实现ReLU算子，然后复用第4.2.4.4节中定义的Linear算子，组建多层感知器MLP_3L。\n",
    "\n",
    "#### 7.6.2.1 ReLU算子\n",
    "\n",
    "假设一批样本组成的矩阵$\\bm Z \\in \\mathbb{R}^{N\\times D}$，每一行表示一个样本，$N$为样本数，$D$为特征维度，ReLU激活函数的前向过程表示为\n",
    "$$\n",
    "\\bm A=\\max(\\bm Z,0)\\in \\mathbb{R}^{N\\times D},\n",
    "$$\n",
    "其中$\\bm A$为经过ReLU函数后的活性值。\n",
    "\n",
    "令$\\delta_{\\bm A}=\\frac{\\partial \\mathcal{R}}{\\partial \\bm A}\\in \\mathbb{R}^{N\\times D}$表示最终损失$\\mathcal{R}$对ReLU算子输出$\\bm A$的梯度，ReLU激活函数的反向过程可以写为\n",
    "$$\n",
    "\\delta_{\\bm Z} =  \\delta_{\\bm A}\\odot(\\bm A>0) \\in \\mathbb{R}^{N\\times D},\n",
    "$$\n",
    "其中$\\delta_{\\bm Z}$为ReLU算子反向函数的输出。\n",
    "\n",
    "下面实现的ReLU算子，并实现前向和反向的计算。由于ReLU函数中没有参数，这里不需要在backward()方法进一步计算该算子参数的梯度。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class ReLU(Op):\n",
    "    def __init__(self):\n",
    "        self.inputs = None\n",
    "        self.outputs = None\n",
    "        self.params = None\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        self.inputs = inputs\n",
    "        return paddle.multiply(inputs, paddle.to_tensor(inputs > 0, dtype='float32'))\n",
    "\n",
    "    def backward(self, outputs_grads):\n",
    "        #计算ReLU激活函数对输入的导数\n",
    "        # paddle.multiply是逐元素相乘算子\n",
    "        return paddle.multiply(outputs_grads, paddle.to_tensor(self.inputs > 0, dtype='float32'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.6.2.2 自定义多层感知器\n",
    "\n",
    "这里，我们构建一个多层感知器MLP_3L。MLP_3L算子由三层线性网络构成，层与层间加入ReLU激活函数，最后一层输出层为线性层，即输出对率（logits）。复用Linear算子，结合ReLU算子，实现网络的前反向计算。初始化时将模型中每一层的参数$\\bW$以标准正态分布的形式进行初始化，参数$\\bm b$初始化为0。函数forward进行网络的前向计算，函数backward进行网络的反向计算，将网络中参数梯度保存下来，后续通过优化器进行梯度更新。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import nndl.op as op\n",
    "\n",
    "class MLP_3L(Op):\n",
    "    def __init__(self, layers_size):\n",
    "        self.fc1 = op.Linear(layers_size[0], layers_size[1], name='fc1')\n",
    "        # ReLU激活函数\n",
    "        self.act_fn1 = ReLU()\n",
    "        self.fc2 = op.Linear(layers_size[1], layers_size[2], name='fc2')\n",
    "        self.act_fn2 = ReLU()\n",
    "        self.fc3 = op.Linear(layers_size[2], layers_size[3], name='fc3')\n",
    "        self.layers = [self.fc1, self.act_fn1, self.fc2, self.act_fn2, self.fc3]\n",
    "\n",
    "    def __call__(self, X):\n",
    "        return self.forward(X)\n",
    "\n",
    "    def forward(self, X):\n",
    "        z1 = self.fc1(X)\n",
    "        a1 = self.act_fn1(z1)\n",
    "        z2 = self.fc2(a1)\n",
    "        a2 = self.act_fn2(z2)\n",
    "        z3 = self.fc3(a2)\n",
    "        return z3\n",
    "\n",
    "    def backward(self, loss_grad_z3):\n",
    "        loss_grad_a2 = self.fc3.backward(loss_grad_z3)\n",
    "        loss_grad_z2 = self.act_fn2.backward(loss_grad_a2)\n",
    "        loss_grad_a1 = self.fc2.backward(loss_grad_z2)\n",
    "        loss_grad_z1 = self.act_fn1.backward(loss_grad_a1)\n",
    "        loss_grad_inputs = self.fc1.backward(loss_grad_z1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.6.2.3 损失函数算子\n",
    "\n",
    "使用交叉熵函数作为损失函数。这里MLP_3L模型的输出是对率而不是概率，因此不能直接使用第4.2.3节实现的BinaryCrossEntropyLoss算子。我们这里对交叉熵函数进行完善，使其可以直接接收对率计算交叉熵。\n",
    "\n",
    "对二分类交叉熵损失进行改写，令向量$\\bm y\\in \\{0,1\\}^N$表示$N$个样本的标签构成的向量，向量$\\bm o\\in \\mathbb{R}^N$表示$N$个样本的模型输出的对率，二分类的交叉熵损失为\n",
    "\n",
    "$$\n",
    "\\mathcal{R}(\\bm y,\\bm o) = -\\frac{1}{N}(\\bm y^{T} \\log \\sigma(\\bm o) + (1 - \\bm y)^{T} \\log(1-\\sigma(\\bm o))),\n",
    "$$\n",
    "\n",
    "其中$\\sigma$为Logistic函数。\n",
    "\n",
    "二分类交叉熵损失函数的输入是神经网络的输出$\\bm o$。最终的损失$\\mathcal{R}$对$\\bm o$的偏导数为:\n",
    "$$\n",
    "\\frac{\\partial \\mathcal R}{\\partial \\bm o} =  -\\frac{1}{N}(\\bm y-\\sigma(\\bm o)).\n",
    "$$\n",
    "\n",
    "损失函数BinaryCrossEntropyWithLogits的代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class BinaryCrossEntropyWithLogits(Op):\n",
    "    def __init__(self, model):\n",
    "        self.predicts = None\n",
    "        self.labels = None\n",
    "        self.data_size = None\n",
    "        self.model = model\n",
    "        self.logistic = op.Logistic()\n",
    "\n",
    "    def __call__(self, logits, labels):\n",
    "        return self.forward(logits, labels)\n",
    "\n",
    "    def forward(self, logits, labels):\n",
    "        self.predicts = self.logistic(logits)\n",
    "        self.labels = labels\n",
    "        self.data_size = self.predicts.shape[0]\n",
    "        loss = -1. / self.data_size * (paddle.matmul(self.labels.t(), paddle.log(self.predicts)) + paddle.matmul((1 - self.labels.t()), paddle.log(1 - self.predicts)))\n",
    "        loss = paddle.squeeze(loss, axis=1)\n",
    "        return loss\n",
    "\n",
    "    def backward(self):\n",
    "        inputs_grads = 1./ self.data_size * (self.predicts - self.labels)\n",
    "        self.model.backward(inputs_grads)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义accuracy_logits函数，输入为logits和labels。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def accuracy_logits(logits, labels):\n",
    "    \"\"\"\n",
    "    输入:\n",
    "        - logits: 预测值，二分类时，shape=[N, 1]，N为样本数量; 多分类时，shape=[N, C]，C为类别数量\n",
    "        - labels: 真实标签，shape=[N, 1]\n",
    "    输出:\n",
    "        - 准确率: shape=[1]\n",
    "    \"\"\"\n",
    "    # 判断是二分类任务还是多分类任务，preds.shape[1]=1时为二分类任务，preds.shape[1]>1时为多分类任务\n",
    "    if logits.shape[1] == 1:\n",
    "        # 二分类时，判断每个logits是否大于0，当大于0时类别为1，否则类别为0\n",
    "        #使用'paddle.cast'将preds的数据类型转换为float32类型\n",
    "        preds = paddle.cast((logits > 0), dtype='float32')\n",
    "    else:\n",
    "        # 多分类时，使用'paddle.argmax'计算最大元素索引作为类别\n",
    "        preds = paddle.argmax(logits, axis=1, dtype='int32')\n",
    "    return paddle.mean(paddle.cast(paddle.equal(preds, labels), dtype='float32'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 7.6.2.4 模型训练\n",
    "\n",
    "使用train_model函数指定训练集数据和测试集数据、网络、优化器、损失函数、训练迭代次数等参数。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def train_model(X_train, y_train, X_test, y_test, model, optimizer, loss_fn, num_iters, *args):\n",
    "    \"\"\"\n",
    "    训练模型\n",
    "    输入：\n",
    "        - X_train, y_train: 训练集数据\n",
    "        - X_test, y_test: 测试集数据\n",
    "        - model: 定义网络\n",
    "        - optimizer: 优化器\n",
    "        - loss_fn: 损失函数\n",
    "        - num_iters: 训练迭代次数\n",
    "        - args: 在dropout中指定模型为训练模式或评价模式\n",
    "    \"\"\"\n",
    "    losses = []\n",
    "    for i in range(num_iters):\n",
    "        # 前向计算\n",
    "        train_logits = model(X_train)\n",
    "        loss = loss_fn(train_logits, y_train)\n",
    "        # 反向计算\n",
    "        loss_fn.backward()\n",
    "        # 更新参数\n",
    "        optimizer.step()\n",
    "        if i % 100 == 0:\n",
    "            losses.append(loss)\n",
    "\n",
    "    train_logits = model(X_train, *args)\n",
    "    acc_train = accuracy_logits(train_logits, y_train)\n",
    "    test_logits = model(X_test, *args)\n",
    "    acc_test = accuracy_logits(test_logits, y_test)\n",
    "    print('train accuracy:', acc_train.numpy())\n",
    "    print('test accuracy:', acc_test.numpy())\n",
    "    return losses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "复用第4.2.4.6节中的BatchGD定义梯度下降优化器。进行50 000次训练迭代，观察模型在训练集和测试集上的准确率。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nndl.op import BatchGD\n",
    "\n",
    "paddle.seed(0)\n",
    "layers_size = [X_train.shape[1], 20, 3, 1]\n",
    "model = MLP_3L(layers_size)\n",
    "opt = BatchGD(init_lr=0.2, model=model)\n",
    "loss_fn = BinaryCrossEntropyWithLogits(model)\n",
    "losses = train_model(X_train, y_train, X_test, y_test, model, opt, loss_fn, 50000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，模型在训练集上的准确率为91%，在测试集上的准确率为71%，推断模型出现了过拟合现象。为了更好地观察模型，我们通过可视化分类界面来确认模型是否发生了过拟合。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可视化函数show_class_boundary的代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def show_class_boundary(model, X_train, y_train, *args, fig_name):\n",
    "    #均匀生成40 000个数据点\n",
    "    x1, x2 = paddle.meshgrid(paddle.linspace(-2, 3, 200), paddle.linspace(-3, 3, 200))\n",
    "    x = paddle.stack([paddle.flatten(x1), paddle.flatten(x2)], axis=1)\n",
    "    #预测对应类别\n",
    "    y = model(x, *args)\n",
    "    y = paddle.cast((y>0), dtype='int32').squeeze()\n",
    "\n",
    "    bg_colors = ['#f5f5f5' if y==1 else '#f19ec2' for y in y]\n",
    "    label_colors = ['#000000' if train_label==0 else '#e4007f' for train_label in y_train]\n",
    "\n",
    "    #绘制类别区域\n",
    "    plt.xlabel('x1')\n",
    "    plt.ylabel('x2')\n",
    "    plt.scatter(x[:, 0].numpy(), x[:, 1].numpy(), c=bg_colors)\n",
    "    plt.scatter(X_train[:, 0].numpy(), X_train[:, 1].numpy(), marker='*', c=label_colors)\n",
    "    plt.savefig(fig_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_class_boundary(model, X_train, y_train, fig_name='opti-regularization.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "图中两种颜色的点代表两种类别的分类标签，不同颜色的区域是模型学习到的两个分类区域。从输出结果看，交界处的点被极细致地进行了区域分割，说明模型存在过拟合现象。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 7.6.3 $ℓ_1$和$ℓ_2$正则化\n",
    "\n",
    "$\\ell_1$和$\\ell_2$正则化是机器学习中最常用的正则化方法，通过约束参数的$\\ell_1$和$\\ell_2$范数来减小模型在训练数据集上的过拟合现象。通过加入$\\ell_1$和$\\ell_2$正则化，优化问题可以写为\n",
    "$$\n",
    "\\theta^{*} = \\mathop{\\arg\\min}\\limits_{\\theta} \\frac{1}{B} \\sum_{n=1}^{B} \\mathcal{L}(y^{(n)}, f(\\bm x^{(n)};\\theta)) + \\lambda \\ell_p(\\theta),\n",
    "$$\n",
    "其中$\\mathcal{L}(\\cdot)$为损失函数，$B$为批量大小，$f(\\cdot)$为待学习的神经网络，$\\theta$为其参数，$\\ell_p$为范数函数，$p$的取值通常为1,2代表$\\ell_1$和$\\ell_2$范数，$\\lambda$为正则化系数。\n",
    "\n",
    "下面通过实验来验证$\\ell_2$正则化缓解过拟合的效果。在交叉熵损失基础上增加$\\ell_2$正则化，相当于前向计算时，损失加上$\\frac{1}{2}\\|\\theta\\|^2$。而反向计算时，所有参数的梯度再额外加上$\\lambda\\theta$。\n",
    "\n",
    "完善算子BinaryCrossEntropyWithLogits，使其支持带$\\ell_2$正则化的损失函数。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class BinaryCrossEntropyWithLogits(Op):\n",
    "    def __init__(self, model, lambd):\n",
    "        self.predicts = None\n",
    "        self.labels = None\n",
    "        self.data_size = None\n",
    "        self.model = model\n",
    "        self.logistic = op.Logistic()\n",
    "        self.lambd = lambd\n",
    "\n",
    "    def __call__(self, logits, labels):\n",
    "        return self.forward(logits, labels)\n",
    "\n",
    "    def forward(self, logits, labels):\n",
    "        self.predicts = self.logistic(logits)\n",
    "        self.labels = labels\n",
    "        self.data_size = self.predicts.shape[0]\n",
    "        loss = -1. / self.data_size * (paddle.matmul(self.labels.t(), paddle.log(self.predicts)) + paddle.matmul((1 - self.labels.t()), paddle.log(1 - self.predicts)))\n",
    "        loss = paddle.squeeze(loss, axis=1)\n",
    "        regularization_loss = 0\n",
    "        for layer in self.model.layers:\n",
    "            if isinstance(layer, op.Linear):\n",
    "                regularization_loss += paddle.sum(paddle.square(layer.params['W']))\n",
    "        loss += self.lambd * regularization_loss / (2 * self.data_size)\n",
    "        return loss\n",
    "\n",
    "    def backward(self):\n",
    "        inputs_grads = 1./ self.data_size * (self.predicts - self.labels)\n",
    "        self.model.backward(inputs_grads)\n",
    "        #更新正则化项对应的梯度\n",
    "        for layer in self.model.layers:\n",
    "            if isinstance(layer, op.Linear) and isinstance(layer.grads, dict):\n",
    "                layer.grads['W'] += self.lambd * layer.params['W'] / self.data_size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "重新训练网络，增加$\\ell_2$正则化后再进行50 000迭代。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "model = MLP_3L(layers_size)\n",
    "opt = BatchGD(init_lr=0.2, model=model)\n",
    "loss_fn = BinaryCrossEntropyWithLogits(model, lambd=0.7)\n",
    "losses = train_model(X_train, y_train, X_test, y_test, model, opt, loss_fn, num_iters=50000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，在训练集上的准确率为86%，测试集上的准确率为77%。从输出结果看，猜测过拟合现象得到缓解。\n",
    "\n",
    "再通过可视化分类界面证实猜测结果，代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_class_boundary(model, X_train, y_train, fig_name='opti-regularization2.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，过拟合现象有所缓解，说明$\\ell_2$正则化可以缓解过拟合现象。\n",
    "\n",
    "### 7.6.4 权重衰减\n",
    "\n",
    "权重衰减(Weight Decay)是一种有效的正则化方法，在每次参数更新时引入一个衰减系数。\n",
    "$$\n",
    "\\theta_t \\leftarrow (1 - \\beta)\\theta_{t-1} - \\alpha \\mathbf g_t,\n",
    "$$\n",
    "其中$\\mathbf g_t$为第$t$步更新时的梯度，$\\alpha$为学习率，$\\beta$为权重衰减系数，一般取值比较小，比如0.0005。\n",
    "\n",
    "完善BatchGD优化器，增加权重衰减系数。定义gradient_descent函数，在参数更新时增加衰减系数。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class BatchGD(Optimizer):\n",
    "    def __init__(self, init_lr, model, weight_decay):\n",
    "        \"\"\"\n",
    "        小批量梯度下降优化器初始化\n",
    "        输入：\n",
    "            - init_lr: 初始学习率\n",
    "            - model：模型，model.params字典存储模型参数值\n",
    "        \"\"\"\n",
    "        super(BatchGD, self).__init__(init_lr=init_lr, model=model)\n",
    "        self.weight_decay = weight_decay\n",
    "\n",
    "    def gradient_descent(self, x, gradient_x, init_lr):\n",
    "        \"\"\"\n",
    "        梯度下降更新一次参数\n",
    "        \"\"\"\n",
    "        x = (1 - self.weight_decay) * x - init_lr * gradient_x\n",
    "        return x\n",
    "\n",
    "    def step(self):\n",
    "        \"\"\"\n",
    "        参数更新\n",
    "        输入：\n",
    "            - gradient：梯度字典，存储每个参数的梯度\n",
    "        \"\"\"\n",
    "        for layer in self.model.layers:\n",
    "            if isinstance(layer.params, dict):\n",
    "                for key in layer.params.keys():\n",
    "                    layer.params[key] = self.gradient_descent(layer.params[key], layer.grads[key], self.init_lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "设置权重衰减系数为0.001。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "model = MLP_3L(layers_size)\n",
    "opt = BatchGD(init_lr=0.2, model=model, weight_decay=0.001)\n",
    "loss_fn = BinaryCrossEntropyWithLogits(model, lambd=0)\n",
    "losses = train_model(X_train, y_train, X_test, y_test, model, opt, loss_fn, num_iters=50000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，训练集上的准确率为84.5%，测试集上的准确率为75%，猜测仍存在过拟合现象，但是现象得到缓解。\n",
    "\n",
    "下面通过可视化分类界面证猜测试结果。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_class_boundary(model, X_train, y_train, fig_name='opti-regularization3.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，权重衰减也可以有效缓解过拟合现象\n",
    "\n",
    "### 7.6.5 暂退法\n",
    "\n",
    "当训练一个深度神经网络时，我们可以随机暂退一部分神经元（即置为0）来避免过拟合，这种方法称为暂退法(Dropout Method)。每次选择暂退的神经元是随机的，最简单的方法是设置一个固定的概率$p$，对每一个神经元都以概率$p$来判定要不要保留。\n",
    "\n",
    "假设一批样本的某个神经层为$\\bm X\\in \\mathbb{R}^{B\\times D}$，其中$B$为批大小，$D$为该层神经元数量，引入一个掩码矩阵$\\bm M \\in \\mathbb{R}^{B\\times D}$，每个元素的值以$p$的概率置为0，$1-p$的概率置为1。\n",
    "\n",
    "由于掩蔽某些神经元后，该神经层的活性值的分布会发生变化。而在测试阶段时不使用暂退，这会使得训练和测试两个阶段该层神经元的活性值的分布不一致，并对之后的神经层产生影响，发生协变量偏移现象。\n",
    "因此，为了在使用暂退法时不改变活性值$\\bm X$的方差，将暂退后保留的神经元活性值放大原来的$1/(1-p)$倍。这样可以保证下一个神经层的输入在训练和测试阶段的方差基本一致。\n",
    "\n",
    "暂退函数$\\mathrm{dropout}$定义为\n",
    "$$\n",
    "\\tilde{\\bm X}=\\mathrm{dropout}(\\bm X) \\triangleq\n",
    "\\begin{cases}\n",
    "(\\bm X \\odot \\bm M)/(1-p) & \\text{当训练阶段时},  \\\\\n",
    "\\bm X & \\text{当测试阶段时}.\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "**提醒**\n",
    "\n",
    "和《神经网络与深度学习》中公式(7.74)不同。两者都可以解决使用暂退法带来的协变量偏移问题，但本书的方法在实践中更常见。\n",
    "\n",
    "在反向计算梯度时，令$\\delta_{\\tilde{\\bm X}}=\\frac{\\partial \\mathcal L}{\\partial \\tilde{\\bm X}}$，\n",
    "则有\n",
    "$$\n",
    "\\delta_{\\bm X}  = \\delta_{\\tilde{\\bm X}} \\odot \\bm M /(1-p).\n",
    "$$\n",
    "\n",
    "这里可以看出，暂退神经元的梯度也为$0$。\n",
    "\n",
    "#### 7.6.5.1 Dropout算子\n",
    "\n",
    "定义Dropout算子，实现前向和反向的计算。注意，Dropout需要区分训练和评价模型。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class Dropout(Op):\n",
    "    def __init__(self, drop_rate):\n",
    "        self.mask = None\n",
    "        self.drop_rate = drop_rate\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        # 生成一个丢弃掩码\n",
    "        mask = paddle.cast(paddle.rand(inputs.shape) > self.drop_rate, dtype='float32')\n",
    "        self.mask = mask\n",
    "        # 随机使一些神经元失效\n",
    "        inputs = paddle.multiply(inputs, mask)\n",
    "        # 使输入的方差保持不变\n",
    "        inputs /= (1 - self.drop_rate)\n",
    "        return inputs\n",
    "\n",
    "    def backward(self, outputs_grad):\n",
    "        return paddle.multiply(outputs_grad, self.mask) / (1 - self.drop_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "定义MLP_3L_dropout模型，，实现带暂退法的网络前反向计算。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nndl.op import MLP_3L\n",
    "\n",
    "class MLP_3L_dropout(MLP_3L):\n",
    "    def __init__(self, layers_size, drop_rate):\n",
    "        super(MLP_3L_dropout, self).__init__(layers_size)\n",
    "        self.dropout1 = Dropout(drop_rate)\n",
    "        self.dropout2 = Dropout(drop_rate)\n",
    "        self.layers = [self.fc1, self.act_fn1, self.fc2, self.act_fn2, self.fc3]\n",
    "\n",
    "    def __call__(self, X, mode='train'):\n",
    "        return self.forward(X, mode)\n",
    "\n",
    "    def forward(self, X, mode='train'):\n",
    "        self.mode = mode\n",
    "        z1 = self.fc1(X)\n",
    "        a1 = self.act_fn1(z1)\n",
    "        if self.mode == 'train':\n",
    "            a1 = self.dropout1(a1)\n",
    "        z2 = self.fc2(a1)\n",
    "        a2 = self.act_fn2(z2)\n",
    "        if self.mode == 'train':\n",
    "            a2 = self.dropout2(a2)\n",
    "        z3 = self.fc3(a2)\n",
    "        return z3\n",
    "\n",
    "    def backward(self, loss_grad_z3):\n",
    "        loss_grad_a2 = self.fc3.backward(loss_grad_z3)\n",
    "        if self.mode == 'train':\n",
    "            loss_grad_a2 = self.dropout2.backward(loss_grad_a2)\n",
    "        loss_grad_z2 = self.act_fn2.backward(loss_grad_a2)\n",
    "        loss_grad_a1 = self.fc2.backward(loss_grad_z2)\n",
    "        if self.mode == 'train':\n",
    "            loss_grad_a1 = self.dropout1.backward(loss_grad_a1)\n",
    "        loss_grad_z1 = self.act_fn1.backward(loss_grad_a1)\n",
    "        loss_grad_inputs = self.fc1.backward(loss_grad_z1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "设置丢弃概率为0.5。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "model = MLP_3L_dropout(layers_size, drop_rate=0.3)\n",
    "opt = BatchGD(init_lr=0.2, model=model, weight_decay=0)\n",
    "loss_fn = BinaryCrossEntropyWithLogits(model, lambd=0)\n",
    "losses = train_model(X_train, y_train, X_test, y_test, model, opt, loss_fn, 50000, 'dev')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，训练集上的准确率为85.5%，测试集上的准确率为76%，猜测仍存在过拟合现象，但是现象得到缓解。\n",
    "\n",
    "通过可视化分类界面证实猜想结果。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_class_boundary(model, X_train, y_train, 'dev', fig_name='opti-regularization4.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从输出结果看，暂退法可以有效缓解过拟合，但缓解效果不如正则化或权重衰减明显。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**动手练习7.9**\n",
    "\n",
    "尝试修改正则化系数、权重衰减系数和暂退概率，观察如果三种参数过大会产生什么现象。\n",
    "\n",
    "## 7.7 小结\n",
    "\n",
    "本章通过动手实现不同的优化器和正则化方法来加深对神经网络优化和正则化的理解。\n",
    "\n",
    "在网络优化方面，首先从影响神经网络优化的三个主要因素（批大小、学习率、梯度计算）进行实验比较来看它们对神经网络优化的影响。为了更好地可视化，我们还进行2D和3D的优化过程展示。除了上面的三个因素外，我们还动手实现了基于随机采样的参数初始化方法以及逐层规范化方法来进一步提高网络的优化效率。\n",
    "\n",
    "在网络正则化方面，我们动手实现了$\\ell_2$正则化、权重衰减以及暂退法，并展示了它们在缓解过拟合方面的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "<div id=\"reference\"></div>\n",
    "\n",
    "# 参考文献\n",
    "[1] 邱锡鹏，神经网络与深度学习，机械工业出版社，https://nndl.github.io/, 2020.   \n",
    "[2] Test functions for optimization. https://en.wikipedia.org/wiki/Test_functions_for_optimization    \n",
    "[3] P. Goyal, P. Dolla ́r, R. B. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. Accurate, large minibatch SGD: training imagenet in 1 hour. CoRR, abs/1706.02677, 2017.     \n",
    "[4] Duchi J, Hazan E, Singer Y, 2011. Adaptive subgradient methods for online learning and stochastic\n",
    "optimization[J]. The Journal of Machine Learning Research, 12:2121-2159.   \n",
    "[5] Tieleman T, Hinton G, 2012. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude[Z].  \n",
    "[6] Kingma D, Ba J, 2015. Adam: A method for stochastic optimization[C]//Proceedings of Interna-\n",
    "tional Conference on Learning Representations.   \n",
    "[7] Ioffe S, Szegedy C, 2015. Batch normalization: Accelerating deep network training by reducing inter-\n",
    "nal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning.\n",
    "448-456.   \n",
    "[8] Ba L J, Kiros R, Hinton G E, 2016. Layer normalization[J/OL]. CoRR, abs/1607.06450. http://arxiv. org/abs/1607.06450.   \n",
    "[9] Hanson S J, Pratt L Y, 1989. Comparing biases for minimal network construction with back-\n",
    "propagation[C]//Advances in neural information processing systems. 177-185."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 附录"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "在实际训练中，我们通过调用飞桨API指定优化器，并通过设置优化器的参数weight_decay=paddle.regularizer.L2Decay(coeff)来实现l2权重衰减正则化。通过飞桨API重复上述实验，构建过拟合情况，实现l2权重衰减正则化，观察过拟合缓解情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class MLP_3L_Paddle(nn.Layer):\n",
    "    def __init__(self, in_features, out_features):\n",
    "        super(MLP_3L_Paddle, self).__init__()\n",
    "        self.fc1 = nn.Linear(in_features, 20)\n",
    "        self.fc2 = nn.Linear(20, 3)\n",
    "        self.fc3 = nn.Linear(3, out_features)\n",
    "        self.act_fn = nn.ReLU()\n",
    "    \n",
    "    def forward(self, X):\n",
    "        z1 = self.fc1(X)\n",
    "        a1 = self.act_fn(z1)\n",
    "        z2 = self.fc2(a1)\n",
    "        a2 = self.act_fn(z2)\n",
    "        z3 = self.fc3(a2)\n",
    "        return z3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def train_model_paddle(X_train, y_train, X_test, y_test, model, optimizer, loss_fn, num_iters):\n",
    "    \"\"\"\n",
    "    训练模型\n",
    "    输入：\n",
    "        - X_train, y_train: 训练集数据\n",
    "        - X_test, y_test: 测试集数据\n",
    "        - model: 定义网络\n",
    "        - optimizer: 优化器\n",
    "        - loss_fn: 损失函数\n",
    "        - num_iters: 训练迭代次数\n",
    "        - args: 在dropout中指定模型为训练模式或评价模式\n",
    "    \"\"\"\n",
    "    losses = []\n",
    "    for i in range(num_iters):\n",
    "        # 前向计算\n",
    "        train_logits = model(X_train)\n",
    "        loss = loss_fn(train_logits, y_train)\n",
    "        # 反向计算\n",
    "        loss.backward()\n",
    "        # 更新参数\n",
    "        optimizer.step()\n",
    "        optimizer.clear_grad()\n",
    "        if i % 100 == 0:\n",
    "            losses.append(loss)\n",
    "\n",
    "    train_logits = model(X_train)\n",
    "    acc_train = accuracy_logits(train_logits, y_train)\n",
    "    test_logits = model(X_test)\n",
    "    acc_test = accuracy_logits(test_logits, y_test)\n",
    "    print('train accuracy:', acc_train.numpy())\n",
    "    print('test accuracy:', acc_test.numpy())\n",
    "    return losses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "paddle.seed(0)\n",
    "model = MLP_3L_Paddle(in_features=X_train.shape[1], out_features=1)\n",
    "opt = paddle.optimizer.SGD(learning_rate=0.2, parameters=model.parameters())\n",
    "loss_fn = F.binary_cross_entropy_with_logits\n",
    "losses = train_model_paddle(X_train, y_train, X_test, y_test, model, opt, loss_fn, 50000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_class_boundary(model, X_train, y_train, fig_name='opti-regularization4.pdf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import paddle.regularizer as regularizer\n",
    "\n",
    "paddle.seed(0)\n",
    "model = MLP_3L_Paddle(in_features=X_train.shape[1], out_features=1)\n",
    "opt = paddle.optimizer.SGD(learning_rate=0.2, parameters=model.parameters(), weight_decay=regularizer.L2Decay(coeff=0.01))\n",
    "loss_fn = F.binary_cross_entropy_with_logits\n",
    "losses = train_model_paddle(X_train, y_train, X_test, y_test, model, opt, loss_fn, 50000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "show_class_boundary(model, X_train, y_train, fig_name='opti-regularization5.pdf')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
